Gemma Team: Researcher — Prolific Citations Library

Explore 1 peer-reviewed study by Gemma Team in LLM and Model Efficiency (2024). Discover research powered by Prolific's participant panel.

This page lists 1 peer-reviewed paper authored or co-authored by Gemma Team in the Prolific Citations Library, a curated collection of research powered by high-quality human data from Prolific.

Papers (1 of 1)

Gemma 2: Improving Open Language Models at a Practical Size

Authors: Gemma Team

Year: 2024

Published in: ArXiv

Institution: Google DeepMind, Google

Research Area: LLM, Model Efficiency, Architecture

Discipline: Artificial Intelligence

Gemma 2 introduces scalable Transformer-based language models (2B-27B parameters) enhanced with techniques like local-global and group-query attention, achieving state-of-the-art performance for their size and competing with larger models.

Methods: The study applied modifications to the Transformer architecture, such as local-global attentions and group-query attention, as well as knowledge distillation training for select model sizes.

Key Findings: Performance of lightweight language models in terms of efficiency and competitiveness with larger models.

Citations: 1649