Title: Theory / Important / CS / Data Science / ML Crash course

Question: Advantage of ML over programming
Answer: Reduce programming time, using off the shelf tool
Page: https://developers.google.com/machine-learning/crash-course/ml-intro
Answer2: Allows to generalize the same tool to multiple data set (e.g. port to other language)
Answer3: Allow to attack program you would not know how to attack
Answer4: Changing the way to see a problem, uses less formal math and more natural science

Name: A batch
Definition: A set of examples used to recalcule the model's loss value and adjusts the model's weights and bias
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression

Name: An iteration
Definition: The process in which a batch is processed and values recalculated
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression

Name: An epoch
Definition: A set of iteration covering each examples
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression

Question: Hyperparameter tuning rules
Answer: Training loss should steadily decrease, steeply at first, and then more slowly until the slope of the curve reaches or approaches zero.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression
Answer2: If the training loss does not converge, train for more epochs.
Answer3: If the training loss decreases too slowly, increase the learning rate. Note that setting the learning rate too high may also prevent training loss from converging.
Answer4: If the training loss varies wildly (that is, the training loss jumps around), decrease the learning rate.
Answer5: Lowering the learning rate while increasing the number of epochs or the batch size is often a good combination.
Answer6: Setting the batch size to a very small batch number can also cause instability. First, try large batch size values. Then, decrease the batch size until you see degradation.
Answer7: For real-world datasets consisting of a very large number of examples, the entire dataset might not fit into memory. In such cases, you'll need to reduce the batch size to enable a batch to fit into memory.

Name: Bucketing feature
Definition: A feature obtaining by transforming a continuous feature into range bins
Page: https://developers.google.com/machine-learning/glossary/#synthetic_feature

Name: Feature cross
Definition: A feature obtained by taking a cartesian product of discrete features
Page: https://developers.google.com/machine-learning/glossary/#synthetic_feature

Name: A synthetic feature
Definition: A feature obtained from other features, by changing them fundamentally
Page: https://developers.google.com/machine-learning/glossary/#synthetic_feature
Example: Bucketing
Example2: Multiplying/dividing features
Example3: feature cross
Counterexample: normalizing feature
Counterexample2: scaling feature

Name: A correlation matrix
Definition: A matrix associating to each feature its importance in the linear regression
Page: https://developers.google.com/machine-learning/glossary/#synthetic_feature
Context: Linear regression

Question: ML basic assumptions in training
Answer: Examples and test are independently and identically at random from the distribution
Page: https://developers.google.com/machine-learning/crash-course/generalization/video-lecture
Answer2: distribution is stationnary through time
Answer3: training validation and test comes from same distribution

Name: training set
Definition: A set of input used to train a model
Page: https://developers.google.com/machine-learning/crash-course/generalization/video-lecture

Name: test set
Definition: A set of input used to test a model
Page: https://developers.google.com/machine-learning/crash-course/generalization/video-lecture

Question: What to do when test is 100% succesful ?
Answer: check you didn't train with test data
Page: https://developers.google.com/machine-learning/crash-course/training-and-test-sets/splitting-data

Question: What problem with reusing a test set multiple time
Answer: You may overfit hyperparameters for the test set
Page: https://developers.google.com/machine-learning/crash-course/training-and-test-sets/splitting-data

Name: Validation data
Definition: A subset of data that are used to fine tune the hyperparamater before looking at final testing
Page: https://developers.google.com/machine-learning/crash-course/validation/video-lecture

Question: Property of good feature
Answer: Features should occur in multiple input (id is okay if the same id appear multiple time)
Page: https://developers.google.com/machine-learning/crash-course/representation/video-lecture
Context: Machine learning
Answer2: Data have sens (e.g. age in year, not in time of birth since epoch)
Answer3: No magic value, instead uses a boolean to check whether value should exists or not
Answer4: Value must not change over time (e.g. use an id by name, so the id remains the same if name change)
Answer5: Values should not have outlier (a house with 50 room/person should be ignored)
Answer6: Binning trick

Question: Tools to know your data
Answer: Visualize: Plot histograms, rank most to least common.
Page: https://developers.google.com/machine-learning/crash-course/representation/video-lecture
Context: Machine learning
Answer2: Debug: Duplicate examples? Missing values? Outliers? Data agrees with dashboards? Training and Validation data similar?
Answer3: Monitor: Feature quantiles, number of examples over time?

Name: Binning trick
Definition: Given some continuous data, transform it in booleans for each interesting range.
Page: https://developers.google.com/machine-learning/crash-course/representation/video-lecture
Context: Machine learning
Example: Use a boolean to indicate whether latitude is the one of san diego
Applied to: Given data about housing price of california and latitude

Question: If the training loss does not converge,
Answer: train for more epochs.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression

Question: If the training loss decreases too slowly,
Answer: increase the learning rate.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression
Extra: Note that setting the learning rate too high may also prevent training loss from converging.

Question: If the training loss varies wildly (that is, the training loss jumps around),
Answer: decrease the learning rate.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression

Question: For real-world datasets consisting of a very large number of examples, the entire dataset might not fit into memory. In such cases,
Answer: you'll need to reduce the batch size to enable a batch to fit into memory.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression

Question: Which batch size to use
Answer: First, try large batch size values. Then, decrease the batch size until you see degradation.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression

Question: What's the danger with too high learning rate?
Answer: prevent training loss from converging.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Context: Linear regression