Goal: Predict a number from a set of features\nOutput: A real number\nProcess: determine optimal parameters by minimizing some average loss and something with a regularization penalty.
\nGoal: Predict a categorical variable(ie win or lose, disease or no disease, spam or ham)\nTypes: Binary Classification(Two classes, ie spam or not spam), Multiclass classication(type of animal)
\nP(Y=1|x) = 1 / (1 + e^((-x^t)_theta)) = sigma(x^T _ theta)\nThe probability of an observation belonging to class 1 given x(vector of features) = Linear regression function on linear transformation of a linear combination of x(vector of features)
\nThus, the logistic regression is sometimes known as a generalized linear model since it is a non-linear transformation of a linear model.
\nCan get trapped in a flat region if the surface is not convex\nSolution: Use Cross-entropy Loss!\nEmpirical Risk for logistic regression when using cross-entropy loss
\nR(theta) = -(1/n)sum from i=1 to n( y_i * log(logistic regression(X_i^T * theta)) + (1 - y_i) * (1 - logistic regression(X_i^T * theta)))
Accuracy = (True positive + True negative) / number of data points
Precision = True positive / (True positive + False positive)
Recall = True positive / (True positive + false negative)
Threshold too high = false negatives, Threshold too low = false positivesThreshold increases = fewer false positives, precision tends to increase(not always)Threshold increases = more false negatives, recall tends to decreaseFalse Positive Rate = False Positives / (False Positives + True Negative)
True Positive Rate = True Positives / (True Positives + False Negatives)
AUC(Area under curve): Area under ROC curve, best possible = 1, worst possible = 0.5(randomly guessing)
\n