Workshop SegFormer: Encoder

Notebook Solution: https://colab.research.google.com/gist/Jeremy26/13f71c273f0a1a93f758d02b2b77802e/segformer-cityscapes-run.ipynb?authuser=0#scrollTo=54M7rPQBQYA1

Next, let's see how the Decoder works. You can try now, and then try again after watching the video if you didn't get how the decoder part works.

SEGFORMERS Exploration

Buy nowLearn more
  • Enter Segformers

Attention & Transformers

  • Introduction to Segformers
  • What is Attention?
  • Recap: The Attention Timeline
  • Other Types of Attention
  • How to Visualize Attention Maps without Code2
  • Transformers
  • Attention & Transformers Q&A3
  • Segformer Networks2
  • Mini-Workshop: Visually Understanding Query, Key, and Value4

Vault: Research Paper Study 🔐

  • Attention is all you need (2017)1
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2021)
  • SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (2021)
  • Karpathy on Transformers

Segformers Workshop

  • The Segformers Starter Block Workshop
  • Block 1 — Overlap Patch Embeddings
  • Block 2 — Efficient Self-Attention2
  • Block 3 — Mix FFNs2
  • Assemble: The Transformer Block2
  • Workshop SegFormer: Part 13
  • Workshop SegFormer: Encoder
  • Workshop SegFormer: Decoder
  • Workshop Segformer: Train & Run10
  • Visualizing the Attention Maps2
  • Bonus: Drivable Area Detection Notebook
  • Hourra!