← Evolution 2.3 · Evolution

Modern Tech Stack

The contemporary data science stack mixes stable foundations — Python, SQL, cloud — with a fast-evolving GenAI layer. Know which is which before choosing where to invest.

📚 4 min read•Updated: October 2025
Stable vs Evolving

Certain foundational technologies have achieved near-universal adoption. The generative AI layer evolves so rapidly that today's best practices may be obsolete within months. Deep expertise belongs to the former; comfort with uncertainty belongs to the latter.

The Full Stack

Four layers, from foundation to frontier

🐍

Python Ecosystem

Python appears in 78% of 2019 data science job postings (57% by 2024 — not declining, just assumed). The most stable choice in the stack.

  • Data manipulation: pandas (standard), Polars (large-scale)
  • Numerical computing: NumPy (foundation for everything else)
  • ML: scikit-learn (classical), PyTorch & TensorFlow (deep learning)
  • Visualisation: matplotlib, seaborn, Plotly (interactive)
  • Apps: Streamlit, Gradio for model interfaces without frontend expertise
🗄️

SQL Resurgence

SQL is now the second-most demanded technical skill. Modern data science increasingly happens "where the data lives" — in cloud warehouses handling terabytes too large to move.

  • Snowflake, BigQuery, Redshift as primary analytics platforms
  • Window functions for sophisticated time-series analysis
  • Common table expressions for readable multi-step transformations
  • JSON functions for semi-structured data
  • Native integration with Python and R for combined workflows
☁️

Cloud Platforms

Cloud transitioned from optional to mandatory — accelerated by the pandemic, driven by economics and capabilities impossible on-premises.

  • Azure: 74.5% of data engineer job postings
  • AWS: 49.5% of data engineer job postings
  • Google Cloud: 21.3% of data engineer job postings
  • Managed services: SageMaker, Azure ML, Vertex AI
  • CapEx → OpEx: pay for hundreds of GPUs for hours, not years
⚙️

MLOps

MLOps matured from conceptual framework to production necessity — bridging the gap between models that work in notebooks and systems that deliver business value at scale.

  • Docker adoption grew 17% YoY amongst data scientists
  • Kubernetes for container orchestration, scaling, and failover
  • MLflow for experiment tracking, model versioning, and metadata
  • Feature stores (Feast, Tecton) ensure training-inference consistency
  • Monitoring for prediction drift, data drift, and fairness
Fast-Evolving Frontier

The GenAI technology layer

This layer evolves so rapidly that specific tools may be superseded within months — yet the categories themselves represent enduring needs.

🧠

Retrieval & Embeddings

  • Vector DBs: Pinecone, Weaviate, Chroma
  • Semantic similarity search
  • RAG pipeline integration
🔗

Orchestration

  • LangChain / LlamaIndex for workflow chaining
  • Conversation history management
  • Tool and API integration
🎯

Ops & Evaluation

  • Prompt management and version control
  • Fine-tuning platforms
  • Output evaluation frameworks
Key Insight — The MLOps infrastructure stack represents the difference between a working model and a production system that delivers business value. It's the "last mile" problem of data science.

Key takeaways

  • Python remains the undisputed foundation despite evolving alternatives
  • SQL resurged as cloud warehouses became the primary analytics platform
  • Cloud computing transitioned from optional to mandatory infrastructure
  • MLOps ecosystem matured to bridge the model-to-production gap
  • GenAI layer evolves rapidly but fundamental categories — retrieval, orchestration, evaluation — remain stable
← Previous: The GenAI Revolution Next: Future Trajectories →