AI-Enhanced Factual Writing

Ref Studio

AI-Enhanced Factual Writing

Services provided

Specialities involved

Services provided

Specialities involved

Project Challenge

Driven by a deep passion for science, our client endeavoured to build a proof of concept, “Ref Studio”, with generative AI to streamline the process of writing research papers, catering to more than just the scientific community. Recognising YLD's expertise in engineering, design, and GenAI, they sought our collaboration based on a shared dedication to open-source projects.

The project presented the challenge of integrating GenAI when it was still a relatively new machine learning tool. Even though it was a new tool and evolving continuously, we successfully developed a highly functional platform, meeting all the requirements to support users in crafting complex scientific papers.

Ref Studio's role as an open-source proof of concept aligns with the evolving dynamics of the scientific community, which is gradually adopting an “open science” approach. This shift is notable among individuals deeply engaged in writing detailed scientific papers.

Our Approach

Functioning as an open-source text editor akin to an Integrated “Development” Environment (IDE) Ref Studio incorporates Large Language Models (LLMs) to offer the following features:

  • Manage references by uploading papers to the app
  • Utilise a chat interface for questions about the content of uploaded reference papers
  • Enable autocomplete and content suggestions while writing
  • Streamline peer collaboration in document editing using a Conflict-Free Replicated Data Type (CDRTs) protocol within a Git-like interface, making it easy to track changes (viewing additions or removals)

Unify the text editor for desktop and web deployment

One of the client's goals was to build a desktop application for Ref Studio through Tauri because it is optimised for performance, it follows modern web development principles, it boasts an active community, and it’s well-documented. We implemented a Python backend, through a sidecar pattern natively supported by Tauri. This backend utilises grobid to parse PDFs and send the references back to the frontend. The accessibility of this function is integrated into the frontend.

As the project progressed, the client also wanted Ref Studio to work on the web. Our team faced a few difficulties since we couldn’t use Tauri’s constructs but we overcame this by updating the Python backend to also be an HTTP server, and by implementing a unified codebase for client/server communication. We also implemented additional APIs for filesystem interactions to take full advantage of a unified codebase for listing, reading, and writing files within the application.

editor

Figure 1: Ref Studio editor

Assist and Chat

The context window in the chat feature is key for a seamless user experience. When posing a question to the GenAI model, providing context is crucial to ensure an accurate output response. In Ref Studio, the model accesses the references (PDF documents) the user has uploaded to the system to respond effectively. The length of the prompt directly impacts the cost, and how the AI sifts through information — the bigger the prompt, the more expensive it is. A longer question on the chat prompts the model to select particular parts from the user's uploaded references, exploring the details of the user's question more thoroughly. The process entails the GenAI model to meticulously select excerpts across the entirety of the uploaded references.

Instead of using the traditional Retrieval-augmented generation (RAG) method, which involves transforming text into vectors and storing them in a database, we opted for a unique approach. This approach still follows the RAG principle but offers the following advantages:

  1. Our approach facilitated fast progress in creating the PoC as it eliminated the need for additional infrastructure complexity like integrating a vector database
  2. A more cost-effective approach to vectors since we didn’t have to host a database and call the AI model to embed text
  3. The process was also faster with no API calls, meaning everything was done locally by the BM25Ranker algorithm

chatbot

Figure 2: Output produced using references

Build an IDE-like application

The client wanted a word processor akin to an Integrated “Development” Environment (IDE). The “word processor” we built, akin to an IDE, allows engineers to streamline code development because it combines functions like software editing, building, and testing within a user-friendly interface that enhances developer productivity. We implemented UI elements for the left and right collapsible sidebars, a footer showing information about the project, and a command palette for quick executions. These elements are common constructs in IDEs which has allowed us to adapt the app to the web environment.

working with references Figure 3: Anatomy of Ref Studio and its functions

The program’s main editor area consists of two editor panes that provide the convenience of tabbed browsing for different types of content. In addition, we implemented comprehensive support for displaying multiple types of content in the main editor area, ensuring a versatile and efficient editing experience. The editor’s main features include:

  • Files that can be edited using a clean and streamlined editor
  • PDF files that the user uploads or gets via Semantic Scholar search
  • Reference details that show metadata extracted from an uploaded PDF or retrieved via Semantic Scholar
  • Reference table for managing the project references, including individual and bulk edits of reference metadata
  • Text file viewer to display text content like markdown and bib files

Implement a clean and streamlined editor using Tiptap

After careful consideration of various text editor libraries, we adopted Tiptap and ProseMirror. Tiptap enabled our team to easily create React components for Ref Studio, while the logic is coded through ProseMirror.

The client wanted us to build an editor that can block browsing, indenting blocks, blocks drag-and-drop, collapsible sections, etc., but TipTap lacked the extensions for these particular features. As a solution, we developed our own extensions by studying ProseMirror's API.

We also wanted to integrate sentence completion directly into Ref Studio to provide a seamless user experience. We achieved this by implementing a new custom extension to display suggestions and enable the user to cycle between them without leaving the editor area. The extension also handles the asynchronicity of the API call and displays an error message in case something goes wrong.

Support local and remote LLM providers

To provide a top-quality feedback experience, Ref Studio supports both non-stream and streaming modes.

In non-stream mode, users can input their text and the model responds with suggestions or completions after processing the entire input. In streaming mode, the model provides real-time feedback instead of waiting for the entire response to be returned, allowing for a more interactive chat experience. The current version uses non-streaming for rewrite and text completion, and streaming for the chat interaction.

In addition to integrating with OpenAI, Ref Studio also offers support for local AI models. Ollama, a local language model provider, can be configured via the application settings, providing users with the option to use AI capabilities even without an internet connection.

Our Impact

We successfully implemented all MVP features within the designated time, presenting a robust proof of concept that lays a strong foundation for future iterations. YLD's proactive approach enabled teams to build a user-friendly and easily maintainable open-source writing environment.

The editor in Ref Studio employed Tiptap and ProseMirror which reflects our dedication to delivering a comprehensive solution. Custom extensions address specific limitations that demonstrate our proficiency in innovating and tailoring solutions to unique requirements. Support for both non-stream and streaming modes, alongside the integration of local AI models like Ollama, highlights our unwavering commitment to providing a top-tier user experience, ensuring accessibility even in the absence of an internet connection.

YLD’s impact on this project extends beyond mere milestones, demonstrating our commitment to adaptability, innovation, and dedication to surpassing expectations. Ref Studio stands as proof to shape the future of scientific writing through the integration of GenAI capabilities.

Featured work


DoctorLink case study preview image

Setting up a Design System for a Cross-Platform Healthcare Product

Setting up a Design System for a Cross-Platform Healthcare Product

A healthcare technology company partnered with YLD to help redesign the user interface and improve upon the UX framework of their cross-platform product. This eventually evolved into the creation of a Design System and the introduction of DesignOps resulting in a cultural shift in their organisation.


Zoa

Navigating B2B and B2C Horizons with React Native

Navigating B2B and B2C Horizons with React Native

Our engineers partnered up with an energy provider resulting in a dynamic app that serves the interests of B2B and B2C clients.


Illustration - Bulb.2

Customisable Energy Management for Every Home

Customisable Energy Management for Every Home

UK energy supplier partnered with YLD for engineering expertise, creating a white-label platform for customer service automation, cost savings prediction, and user-friendly household energy management, transactions, and bill splitting.

Find us

London - HQ

9 Dallington Street

London

EC1V 0LN

+44(0) 203 514 4678

hello@yld.io

Lisbon

Rua Ramalho Ortigão 8

3º Esquerdo

1070-230

Lisboa

Porto

Rua Sá da Bandeira 819

2º Esquerdo

4000-438

Porto