The paper introduces the fast feedforward (FFF) architecture, which offers comparable performance to feedforward networks at a fraction of the inference cost. FFFs can replace feedforward networks and mixture-of-expert networks in transformers. A vision transformer trained with FFF achieves single-neuron inferences with only a 5.8% performance decrease.

https://arxiv.org/abs//2308.14711

YouTube: https://www.youtube.com/@ArxivPapers

PODCASTS:
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

Ver todos los episodios