Our Work | Notion

Understanding Objectivity of LLM Outputs: Pre-, Mid-, and Post-hoc Approaches

Interested?

Innovative linear time scaling architectures such as RWKV, Mamba, and RetNets provide huge opportunities to reduce LLM-based emissions, speed them up significantly, and support “infinite” context lengths. A future avenue of research is to review these architectures, analyzing their performance at different scales with different training sets.
Along this line of accelerated language modeling, hardware language model accelerators provide significant potential to further boost speeds. Work by companies such as Groq open up the avenue for running large language models at hundreds, perhaps thousands of tokens per second. Review different architectural paradigms to accomplish this, alongside speculating on future directions remains an open avenue.
Architectures such as tree-of-thought, beam search, and language agent tree search provide huge upside potential in augmenting LLM reasoning. However, these architectures are often hard to implement. We aim to implement these architectures in an easy to use, extensible package to allow for further LLM reasoning improvements.
Music generation has been a historically difficult problem; models are able to generate music in the likes of MP3 or WAV files, which, while useful, are not extensible. We aim to explore how sequence to sequence models can be used to generate MIDI music, which can be changed, edited, and built on top off by prospective musicians.