Course curriculum

  • 1

    Module 1: Introduction to Language Models

    • What Are Language Models (and Why Should You Care)?

    • The Rise of Small Language Models

    • SLMs vs. LLMs: Speed, Scale, and Smarts Compared

    • Where SLMs Shine: Real-world Use Cases (Edge AI, Privacy-first Apps, Offline Assistants)
  • 2

    Module 2: Core Concepts in NLP and Model Architecture

    • Tokenisation and Vocabulary Design

    • Transformer Architecture: The Essentials

    • Attention Mechanisms Explained Simply

    • How Size Affects Performance, Speed, and Cost
  • 3

    Module 3: Training Small Language Models

    • Dataset Preparation and Cleaning

    • Choosing Model Size and Architecture (parameter budgets)

    • Training Objectives: Masked, Causal, and Instruction-based

    • Techniques for Efficient Training (Quantisation, Pruning, LoRA)
  • 4

    Module 4: Evaluation and Benchmarking

    • Measuring Performance: Perplexity, BLEU, and Beyond

    • Evaluating SLMs for Task-Specific Performance

    • Common Pitfalls in Small Model Evaluation

    • Ethical and Bias Considerations in Compact Models
  • 5

    Module 5: Deployment and Optimisation

    • On-device Inference (Edge, Mobile, and IoT)

    • Model Compression and Distillation

    • Latency, Memory, and Power Optimisation

    • Serving SLMs in Production (APIs, Containers, and Pipelines)
  • 6

    Module 6: Usecases of SLMS

    • Chatbots and Personal Assistants

    • Domain-Specific SLMs (e.g. Medical, Legal, Finance)

    • Multilingual and Low-resource Adaptations

    • Hybrid Systems: Combining SLMs with Retrieval or Rules
  • 7

    Module 7: Advanced Topics

    • Fine-tuning and Continual Learning

    • Instruction-tuning and Prompt Engineering for SLMs

    • Open-weight Models (Phi, TinyLlama, Mistral)

    • Future Directions in Compact AI
  • 8

    Study Material

    • Study Material