Mixture of In-Context Learners

Uses subsets of demonstrations to train experts via in-context learning. Given a training set, a trainable weighting function is used to combine the experts' next-token predictions.

This approach applies to black-box LLMs since access to the… pic.twitter.com/nhQRwYrVjz

— elvis (@omarsar0) November 6, 2024

Pretty cool!! Lets #connect