Traditional loops over lists become a major performance bottleneck when processing large amounts of data.
Learn the fundamentals of linear algebra, where data points are vectors, datasets are matrices, and NumPy arrays provide the data structure.
The k-nearest neighbors algorithm classifies data points by finding the most common label among their closest neighbors in a vector space.
A walkthrough of an unoptimized k-nearest neighbors implementation demonstrates the severe performance issues caused by nested loops.
Replace the inner loop for calculating Manhattan distance with a single vectorized subtraction operation on NumPy arrays.
Use array reshaping and broadcasting to perform all pairwise distance calculations simultaneously, avoiding explicit replication and nested loops.
Gain final performance improvements by replacing Python's list sorting with NumPy's faster sorting algorithms and efficient array slicing.
NumPy arrays are faster because they store data in a contiguous block of memory, which is more efficient for the CPU to process than the scattered memory of Python lists.
Answers common questions about NumPy's underlying C implementation, hyperparameter tuning, memory management, and career paths in data science.
Bonial International GmbH
Berlin, Germany
56:46Jodie Burchell
59:43Paul Graham
21:02Andy Terrel
46:30Anirudh Koul
57:46Adrian Schmitt
50:38Radovan Kavický
39:04Bobur Umurzokov
33:44Fabian Schindler
Jobs that call for the skills explored in this talk.
Corriculo Ltd
Oxford, United Kingdom
Rolecode
Northampton, United Kingdom
Resourceful Talent Group
Vienna, Austria
Rolecode
Cambridge, United Kingdom
SIAL TECHNOLOGY PARTNERS LLC
Nashville, United States of America
Resourceful Talent Group
Amsterdam, Netherlands
