NVIDIA Machine Learning Engineer Interview Questions

I would choose gradient boosted trees when the data is tabular, the dataset is not huge, and I need strong performance with relatively little feature engineering overhead. They are often easier to train, faster to iterate on, and more interpretable than deep networks for structured data. The reason I choose this approach is that the best model is usually the one that fits the data and constraints, not the fanciest one. I would also consider deployment cost, latency, and how much tuning effort the team can support. If the interviewer wants more detail, I would compare this choice against a deep model on GPU-heavy workloads.

I would first profile the pipeline to see whether the bottleneck is data loading, kernel efficiency, memory movement, or communication between devices. Then I would look at batch size, mixed precision, input pipeline throughput, and whether the model is underutilizing the GPU because of small kernels or CPU overhead. The reason I choose this approach is that GPU inefficiency usually comes from one or two specific stages rather than the whole training loop. I would also think about overlapping compute with transfer, using fused ops, and scaling out with the right parallelism strategy. If needed, I would explain how I would validate each change with throughput and utilization numbers.

I would start with the model size, memory footprint, and communication cost. Data parallelism is usually the simplest when the model fits on each device, while tensor parallelism helps when individual layers are too large for one GPU, and pipeline parallelism helps when the model is deep enough to split into stages. The reason I choose among them based on constraints is that no single parallelism strategy is best for every model. I would also think about hardware topology, synchronization overhead, and how much engineering complexity the team can tolerate. If the interviewer wants more depth, I would mention that hybrid strategies are common for very large models.

I would first inspect the learning rate, gradients, loss scaling, and whether the issue appeared after a recent code or data change. Then I would look for unstable operations such as division by zero, overflow, bad normalization, or mixed-precision issues. The reason I choose this debugging path is that NaNs often come from a small numerical instability that becomes visible only after several steps. I would also think about logging intermediate activations and checking whether the input batch contains corrupted examples. If needed, I would explain how I would bisect the training run to isolate the failing step.

I would first measure whether the latency comes from prompt processing, model execution, memory bandwidth, or output generation. Then I would consider batching, quantization, smaller models, caching, and kernel optimization depending on the bottleneck. The reason I choose this approach is that LLM latency is usually a systems problem as much as a model problem. I would also think about request deadlines, streaming responses, and whether throughput or tail latency matters more for the product. If the interviewer wants more detail, I would discuss how GPU utilization and queueing affect end-to-end latency.