Title: Theory / Unimportant / CS / Programming / Python / SciPy / SciPy
Tags: automatedNewLineAfterCommaSpace

Name: SciPy library
Definition: a collection of mathematical algorithms and convenience
functions built on the NumPy extension of Python
Page: https://docs.scipy.org/doc/scipy/reference/tutorial/general.html

Name: Special package
Definition: definition of
numerous special functions of mathematical physics. Available
functions include airy, elliptic, bessel, gamma, beta, hypergeometric,
parabolic cylinder, mathieu, spheroidal wave, struve, and
kelvin
Page: https://docs.scipy.org/doc/scipy/reference/tutorial/general.html
Context: SciPy library

Name: Integrate
Definition: sub-package provides several integration
techniques including an ordinary differential equation integrator
Page: https://docs.scipy.org/doc/scipy/reference/tutorial/general.html
Context: SciPy library

Question: What does return integration functions
Answer: Pair with the result and the error bound
Page: https://docs.scipy.org/doc/scipy/reference/tutorial/general.html
Context: SciPy library

Instruction: integrate.simps(
  [y_1,
  ...,
  y_n],
  [x_1,
  ..., x_n]
)
Page: https://docs.scipy.org/doc/scipy/reference/tutorial/general.html
Language: SciPy
Returns: The integral of a function with value y_i at position x_i
Assuming: Polynomial of degree at most 2,
  or evenly spaced \(
  2^k+1\
) samples and degree at most 3

Name: Optimize
Definition: several commonly used
optimization algorithms
Page: https://docs.scipy.org/doc/scipy/reference/tutorial/integrate.html
Context: SciPy library

Instruction: minimize(
  f,
  init
)
Page: https://docs.scipy.org/doc/scipy/reference/tutorial/integrate.html
Context: SciPy library Optimize
Returns: A local minima of f, found starting from init
Variables: f a funciton taking an array of n float
init an array of n floats

Name: Supervised machine learning
Definition: ML systems learn how to combine input to produce useful predictions
on never-before-seen data.
Page: https://developers.google.com/machine-learning/crash-course/framing/ml-terminology

Name: Label
Definition: the thing we're predicting
Page: https://developers.google.com/machine-learning/crash-course/framing/video-lecture
Context: Machine Learning
Example: y
Example2: future price
Example3: spam or not
Applied to: Linear regression

Name: Feature
Definition: An input variable
Page: https://developers.google.com/machine-learning/crash-course/framing/video-lecture
Context: Machine Learning
Example: x
Name2: An unlabeled example
Example2: An email content
Applied to: Linear regression
Applied to2: Spam detection

Name: Labeled example
Definition: An input variable and a label
Page: https://developers.google.com/machine-learning/crash-course/framing/video-lecture
Context: Machine Learning

Name: An example
Definition: A labeled or unlabeled example
Page: https://developers.google.com/machine-learning/crash-course/framing/ml-terminology
Context: Machine Learning

Name: Training
Definition: creating (learning) the model
Page: https://developers.google.com/machine-learning/crash-course/framing/ml-terminology
Context: Machine Learning

Name: Inference
Definition: applying the trained model to unlabeled examples
Page: https://developers.google.com/machine-learning/crash-course/framing/ml-terminology
Context: Machine Learning

Name: Regression
Definition: Predicting of continuous values
Page: https://developers.google.com/machine-learning/crash-course/framing/ml-terminology
Context: Machine Learning
Example: House value

Name: Classification
Definition: Predicting of discrete values
Page: https://developers.google.com/machine-learning/crash-course/framing/ml-terminology
Context: Machine Learning
Example: spam or not
Example2: Dog, cat or hamster

Name: Loss
Page: https://developers.google.com/machine-learning/crash-course/descending-into-ml/video-lecture
Context: Machine Learning
Intuition: Difference between predicted label and true value

Name: L_2Loss
Definition: \(\sum_{(x,y)\in D}(y-\mathtt{prediction}(x))^2\)
Page: https://developers.google.com/machine-learning/crash-course/descending-into-ml/video-lecture
Context: Machine Learning

Name: Mean square error
Definition: \(\frac{1}{|D|}\sum_{(x,y)\in D}(y-\mathtt{prediction}(x))^2\)
Page: https://developers.google.com/machine-learning/crash-course/descending-into-ml/video-lecture
Context: Machine Learning
Abbreviation: MSE
Intuition: Loss function which penalize having big errors more than having few small errors

Name: Goldilocks learning rate
Definition: The optimal learning rate
Page: https://developers.google.com/machine-learning/crash-course/fitter/graph
Context: Machine Learning

Name: hyperparameter
Definition: A paramater used for training the model, and not a parameter we are trying to optimize/find
Page: https://developers.google.com/machine-learning/crash-course/fitter/graph
Context: Machine Learning
Example: Size of the steps in gradiant descents

Question: What's the main problem with gradiant descent?
Answer: There are local minimum that are not global
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/video-lecture
Context: Machine Learning

Name: A model has converged
Definition: A model is in a state where progress does not occur or where progress is sufficiently slow
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/video-lecture
Context: Machine Learning

Name: The gradiant of \(f\) at \(\mathbf y\)
Definition: \(\left(\partial f\over \partial {x_i}(\mathbf y)\right)_{1\le i\le n}\)
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/gradient-descent
Variables: \(f:\mathbb R^n\to\mathbb R\)
\(\mathbf y\in\mathbb R^n\)
Context: Machine Learning
Notation: [$]\grad f(\mathbf y)[/$]
Intuition: The direction of the greatest increase of \(f\)

Name: Learning rate
Definition: An hyperparameter by which gradiant are multiplied to choose a step in learning algorithm
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/learning-rate
Context: Machine Learning
Name2: Step size

Question: Ideal learning rate of one dimensional function
Answer: \(1\over{f(x)''}\)
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/learning-rate
Context: Machine Learning

Question: Ideal learning rate of 2 dimensional function
Answer: Inverse of the hessian
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/learning-rate
Context: Machine Learning

Name: Batch size
Definition: Total number of examples
you use to calculate the gradient in a single iteration.
Page: https://colab.research.google.com/github/google/eng-edu/blob/main/ml/cc/exercises/linear_regression_with_synthetic_data.ipynb
Context: Machine Learning batch

Name: Stochastic gradient descent
Definition: Gradient descent with batch of size 1
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/learning-rate
Context: Machine Learning batch
Abbreviation: SGD

Name: Mini-batch stochastic gradient descent
Definition: Gradient descent with batch of size 10 to 1000
Page: https://developers.google.com/machine-learning/crash-course/reducing-loss/learning-rate
Context: Machine Learning batch

Name: Tensorflow
Definition: an end-to-end open source platform for machine learning.
Page: https://developers.google.com/machine-learning/crash-course/first-steps-with-tensorflow/toolkit
Abbreviation: tf