Statistical language models are central to many applications that use semantics. Recurrent Neural Networks (RNN) are known to produce state of the art results for language modelling, outperforming ...

We present the results of experiments on minimizing the model size for the text-based Open Vocabulary Keyword Spotting task. The main goal is to perform inference on devices with limited computing ...

The AI research community continues to find new ways to improve large language models (LLMs), the latest being a new architecture introduced by scientists at Meta and the University of Washington ...

I believe that having a few vocabulary types for very common needs in HPC applications, based on View and Array which are the bread and butter of Kokkos data structures, would greatly benefit the ...

Current state-of-the-art models for natural language understanding require a preprocessing step to convert raw text into discrete tokens. This process known as tokenization relies on a pre-built ...

Cross-Architecture Adaptation: Using mergekit-tokensurgeon, a version of Qwen2.5-14B was created that uses the vocabulary of Llama 3.1 405B. This allowed for the use of Llama 3.1 405B logits in ...

To address this, researchers have proposed a straightforward and effective method to scale up the vision vocabulary for LVLMs by training a new visual vocabulary network using a smaller ...