Stochastic Gradient Descent

• Gradient Descent (Batch): You take a step in the steepest downhill direction. To find the steepest direction, you have to survey the slope of the entire landscape (the entire dataset) before taking each single step. This is accurate but very slow if the landscape is vast (a huge dataset).Stochastic 
• Gradient Descent (SGD): Instead of surveying the entire landscape, you just pick one random spot on the landscape and measure the slope there. You then take a small step in that single spot's steepest downhill direction. You repeat this process many times, picking a new random spot for each step.