Machine learning models use gradient descent to optimize their loss function during training. The backpropagation algorithm computes gradients efficiently by applying the chain rule. Modern architectures like transformers use attention mechanisms to process sequential data, enabling better performance on NLP tasks.