Review of AI Large Language Models’ Implementation for Java

AI based on an LLM (large language model) has increasingly been introduced into various branches of modern life. Many famous IT companies have joined the AI race. Training of large language models takes a lot of time and resources. The training process is usually divided into two stages: pretraining and fine-tuning. In the first stage, […]

Category

Technologies

Posted

Eugene

May 15, 2025

AI based on an LLM (large language model) has increasingly been introduced into various branches of modern life. Many famous IT companies have joined the AI race.

Training of large language models takes a lot of time and resources. The training process is usually divided into two stages: pretraining and fine-tuning. In the first stage, models are trained without a teacher on large amounts of data. For this purpose, using materials which consisting of books, articles, news, Wikipedia, etc.

Common Crawl is one of the well-known datasets containing a lot of texts from Internet pages, which is often used at the pre-training stage. At the setup stage, already labeled datasets are used to train the model to solve different problems. Moreover, the number of such datasets can reach several hundred for one model. Many companies pay special attention to the cleanliness of the data on which they train their models, so it is not surprising that most of the time, the cleaned data remains inaccessible, even if the model itself is publicly available.

 Java world has implemented several LLM solutions. Let’s review some of them.

Hexascribe (Chatbot-Builder, VertexAI-kt)

The PaLM 2 model included in Vertex is a competitor for GPT3, but also GPT4. It also includes Imagen (generates images). It has several implementations. The first of them provides API up to Palm 2, which has published under Apache 2.0 license (can be used in a commercial application) and located on GitHub at hexascribe/vertexai-kt. It is adapted for Kotlin Multiplatform,  so it can be used not only on Android, but also on iOS. If you want to create chatbots, there is a second lib for this – hexascribe/chatbot-builder.

Langchain4j

The essence of the langchain is a competent set of abstractions on top of which unified chains of transformations are built that can be connected to different data sets. Since October 2022, an entire ecosystem of components and modules has been built around it – from the obvious API for models to persistence and streaming of different stages of the chain. You can take some initial piece of data, and then make 100500 “offshoots” with different processing methods. For example, wrap it with prompt templates, send it to several cloud LLMs (GPT3, BLOOM), save the results in different subsystems. langchain4j/langchain4j is published under Apache 2.0 license, and is a dumb Java wrapper. It will not interfere with the logic of the ecosystem, and if necessary, you can easily add the missing functionality yourself.

Chat2DB

The following library turns spoken language into SQL. That is, it is a client to an SQL database, with support for MySQL, PostgreSQL, Oracle. There is a smart assistant that helps with query optimization. The chat2db/Chat2DB library is posted on GitHub under the Apache 2.0 license, and even has documentation in English. The project was created in the depths of Alibaba. The documentation even describes how to connect from China via VPN to Open AI services.

Llama2j

It is Java implementation of standalone LLama 2 inference – Meta LLM product which is alternative with GPT-4o, Claude-3.5, Sonnet.

Future Predictions

Some experts believe that the creation of LLMs can be more progressive in the next 5 years. This may well be true, and there are several prerequisites for this.

First, the computing power of neural networks is beginning to compare with human ones, and the models are becoming multimodal, allowing them to learn from different types of data.

Second, quantity must sooner or later turn into quality. The number of LLMs and their sizes are increasing, the quality of generated text is approaching that of human writing, and there is a possibility that in the foreseeable future a certain threshold will be reached at which LLMs will have to be created simply mathematically.

Third, OpenAI has already mentioned the existence of a Q* model that can solve school math problems, which is another step toward AI.

If you need to take a step toward connecting with a great technology partner to help with your AI or other tech needs, contact Swan Software Solutions to schedule a free assessment.