Semantic search with embeddings in Spring & Kotlin
This article is aimed at application developers who are curious and looking for an uncomplicated entry into modern AI approaches without having to leave their familiar tech stack. The focus is on embeddings and shows in a practical way how AI functionality can be seamlessly integrated into existing systems. I deliberately chose Kotlin for the examples. The language allows a concise and pragmatic syntax with which the concepts can be conveyed in a compact and understandable way. There is also a Java version in the GitHub repository.
Data has long been known as the new gold. But until now, much of it was inaccessible or considered unusable. Texts, images and other unstructured content lay idle, with no efficient way to search or analyze it in a meaningful way. Embeddings have radically changed this. They transform words, sentences and even images into mathematical vectors that not only capture character strings, but also their meaning. This opens up completely new possibilities such as semantic search, automatic categorization and personalized recommendations. In addition, all progress in the field of generative AI and in particular large language models is based on embeddings.
Embeddings enable machines not only to read content, but also to understand it. They enable AI systems to link content in the same way that humans do, by capturing not only the structure but also the meaning. Those who understand this fundamental way of functionality can create innovations that were previously unthinkable. Old-school software developers in particular often believe that this technology is just a trendy extra feature or something for data scientists and AI enthusiasts. This is definitely not the case, as embeddings have become the fundamental basis of modern software development in recent years!
What exactly are embeddings or word vectors?
What exactly are embeddings or word vectors?
Embeddings are mathematical representations of MEANING. For machines, meaning was previously a completely abstract concept, because semantics is something that humans tend to use intuitively without explicitly defining it. However, embeddings have made it possible to map the meaning of words, sentences, images or other unstructured data into numbers, i.e. vectors. This means that such data can now be processed automatically and its content understood.
#The word “car” as a vector:
[ -0.024003787, -0.014898447, 0.028530063, -0.001280026,
0.0040512164, -0.06925336, -0.024373278, 0.007330458,
-0.032911185, -0.021839619, 0.028978733, -0.012476955...]A simple example is the semantic search in a product database. Classic search algorithms are usually based on keywords. Anyone searching for “car” must use this word in the query. If “vehicle” is entered instead or a spelling mistake is made by mistake, the search is unsuccessful. Embeddings work differently. A semantic search recognizes connections between words and also finds terms that do not contain the exact word being searched for. Relevant synonyms, misspellings and similar terms are taken into account so that search results no longer depend on exact formulations.
The concept of word vectors
Embedding technologies such as Word2Vec, GloVe or FastText represent words as vectors. A vector is basically nothing more than an array of decimal numbers, and these can easily be represented in a graph to analyze relationships between vectors or calculate similarities. For example, semantic relationships between words or sentences can be visually captured and mathematically compared.
These word vectors capture semantic similarities by placing terms with similar meanings closer together in the vector space. In this way, models can recognize that “king” and “queen” have a similar relationship to each other as “man” and “woman”.
But one problem remains with this approach: The meaning of a word can vary greatly depending on the context. The word “bat” can refer to both a flying mammal and a piece of sports equipment used in baseball. Classic word vectors were unable to capture this difference, as they only assigned one fixed representation to each word.
Context embeddings
Modern context embeddings such as BERT or GPT solve this problem by representing not only individual words, but entire sentences or paragraphs as high-dimensional vectors. Word meanings are captured in direct connection with their environment. The model understands that “bat” has a different meaning in “A bat is flying in the night sky” than in “He hit the ball with a bat”.
A multidimensional vector space is like a huge room with many invisible axes in which words, sentences or entire texts exist as points. Each dimension of this space stands for a specific property or meaning. For example, one dimension can stand for “state”, another for “emotion” and another for “tense”. The more dimensions there are, the more accurately a model can capture meanings. Thus, Word2Vec models typically create word vectors with 100 to 300 dimensions, whereas the OpenAI model text-embedding-3-large creates vectors with 3,072 dimensions by default.
Similar terms are not only close to each other, but also in certain directions that show how they relate to each other. In this way, a model can not only recognize that “bank” is used differently in a financial context than in a seating context, but also which other words or phrases fit thematically.
We build a semantic search
So far for the theory. Now we will implement a semantic search using a concrete example to better understand the fascinating world of embeddings.
Our PoC should make it possible to search for any song using search terms or descriptions. For example, users should be able to find suitable songs if they enter terms such as “love songs”, “Queen” or “I’m sad”.
Our roadmap
The PoC is developed as a REST service in Kotlin and Spring and uses Spring AI. In addition to a simple web interface, the focus is on a REST service that implements the actual similarity search. For this, we need an embedding model and a vector database. At this point, we rely on the VectorStore with the SimpleVectorStore implementation of Spring AI. We will generate the required song data using a GPT model.
Of course, the example could also be implemented in Python or TypeScript. But especially in the enterprise environment and in classic application development, where Java, C# or similar ecosystems are used, for example, I believe that an understanding of GenAI topics should be specifically promoted.
Thanks to Spring AI, various GenAI models and approaches can now be used very easily in the Java environment. I think that Kotlin in combination with Spring AI is a great tech stack. So, let’s get started!
Spring project with Spring-AI
We create a simple Spring project with Kotlin, Gradle and spring-boot-starter-web. We can either use the integrated functionality of IntelliJ IDEA or the Spring Initializr. We use version 21 of the JDK.
After we have created the project, we add the following spring-ai dependency to build.gradle.kts:
implementation("org.springframework.ai:spring-ai-transformers-spring-boot-starter")With spring-ai, however, we have to take a few things into account. The dependency also requires the corresponding spring-ai-bom, which we have to import via the dependencyManagement plugin. In the end, our build.gradle.kts looks something like this:
plugins {
...
id("io.spring.dependency-management") version "1.1.7"
}
dependencies {
...
implementation("org.springframework.ai:spring-ai-transformers-spring-boot-starter")
}
dependencyManagement {
imports {
mavenBom("org.springframework.ai:spring-ai-bom:1.0.0-M6")
}
}We have thus integrated all the necessary dependencies for Spring AI.
The package spring-ai-transformers-spring-boot-starter enables the integration of ONNX transformer embedding models in Spring Boot applications. The great advantage of ONNX models is that they are technology-agnostic. This means that they can be used not only in Java, but also in other ecosystems such as Python and TypeScript. This interoperability makes ONNX a flexible and cross-platform solution that can be used in different development environments.
EmbeddingsService
Now we create a service called EmbeddingService and implement the embed method, which creates a vector from a simple string:
import org.springframework.ai.embedding.EmbeddingModel
@Service
class EmbeddingsService( val model: EmbeddingModel ) {
fun embed(text: String) = model.embed(text)
}It is important that we inject the EmbeddingModel dependency into the EmbeddingService. The package spring-ai-transformers-spring-boot-starter provides all the necessary components, including a standard embedding model (all-MiniLM-L6-v2). This allows us to get started right away without having to make additional configurations, download models or register for premium APIs and obtain API keys. This is because the models used here are open source and run directly in the local process and memory without having to access external APIs.
EmbeddingsController
Now just create the EmbeddingsController and route through the embed method of the EmbeddingService and you’re done!
@RestController
class EmbeddingsController(val embeddings: EmbeddingsService) {
@GetMapping("/embeddings")
fun embed(@RequestParam text: String) = embeddings.embed(text)
} Now we can start the service, the corresponding URL:
http://localhost:7070/embeddings?text=test
in the browser and marvel at our freshly created vectors! After calling up the URL, the following response should come back:
[ 0.19300838,0.405951,-0.40843546,-0.0717052,-0.019136434... ]Vector databases
Embeddings are primarily data that can be searched or compared with each other. If required, this vector data can be loaded into memory and analyzed using various algorithms, for example to enable a semantic search. However, vector databases exist specifically for such tasks. They are optimized to store vectors efficiently and perform fast similarity comparisons. This makes them particularly suitable for semantic searches, text classification, retrieval augmented generation (RAG) and many other AI-based approaches.
VectorStore
The VectorStore in Spring AI is an abstract construct that enables the use of various vector databases. One of these implementations is the SimpleVectorStore, a simple yet efficient in-memory vector database. It allows vectors to be loaded, quickly searched and persisted to the file system as needed without relying on complex external databases or APIs. To do this, we create a simple configuration with the name VectorStoreConfig and register the corresponding bean at this point.
@Configuration
class VectorStoreConfig {
@Bean
fun vectorStore(model: EmbeddingModel): VectorStore =
SimpleVectorStore.builder(model).build()
}Now we can implement the final method for the semantic search in the EmbeddingsService.
@Service
class EmbeddingsService(val model: EmbeddingModel, val store: VectorStore) {
fun embed(text: String) = model.embed(text)
fun similar(text: String) = store.similaritySearch(text)
}And now just extend the controller and pass through the corresponding similarity search method in the controller:
@RestController
class EmbeddingsController(val embeddings: EmbeddingsService) {
@GetMapping("/embeddings")
fun embed(@RequestParam text: String) = embeddings.embed(text)
@GetMapping("/similarity")
fun similar(@RequestParam text: String) = embeddings.similar(text)
}Now start the service again and call up the following URL: http://localhost:7070/similarity?text=test
And we notice that we are not getting any results. Our response should currently look like this:
[]This is correct, because although we have integrated our vector database and implemented the similarity search, the database is missing data because it is empty.
Data
First of all, we need a solid database. A quick reminder: We want to implement a semantic song search and therefore need song data. We could search for existing data, but we would have to keep an eye on copyrights. Alternatively, we could laboriously create the data ourselves, i.e. research and formulate around 100 song descriptions, which would take a lot of time. Or we could use the GenAI approach and have a GPT model generate synthetic data conveniently and in compliance with the GDPR. The creation of synthetic data is one of the great revolutions that progress in GenAI has made possible and this is exactly what we are taking advantage of.
I use the following prompt for this:
Create a music database with 100 song descriptions.
Each description should consist of a single sentence and a maximum of
200 words. It should include both the song title and the artist name.
Briefly describe the theme of the song and the mood (e.g. happy,
melancholic, energetic etc.). The song selection should be varied:
different artists, genres and decades to ensure broad diversity.
Each description should be separated by a line break.
There should be no listings or numbering, only the description.
Please create all 100 descriptions directly without asking.We can now use this prompt with ChatGPT or another LLM-based chat, for example. I personally prefer Perplexity.ai at this point
You should get roughly the following result.
"Let It Be" von The Beatles ist eine tröstende Ballade...
"Hurt" von Johnny Cash ist eine tief bewegende ...
"Shake It Off" von Taylor Swift ist ein fröhlicher Pop-Song ..
...We save the result in the text file songs.raw.en.txt. As we use simple embedding models in our PoC, these deliver good results, especially in English.
Loading raw data into the VectorStore
Now that we finally have our song descriptions in the text file songs.raw.en.txt, we can vectorize this data and fill the vector database with it. To do this, we extend the existing EmbeddingsService. This gets an additional method createVectorData, which is executed immediately when the service is started thanks to the @PostConstruct annotation and fills our VectorStore with the corresponding data.
...
@Value("classpath:data/songs.raw.en.txt")
private val songsRes: Resource? = null
@PostConstruct
fun createVectorData() {
val docs = songsRes!!.getContentAsString(StandardCharsets.UTF_8)
.split("\n")
.filter { it.isNotBlank() }
.map { Document(it) }
store.add(docs)
}
...We store the data itself, i.e. the text file songs.raw.en.txt, in the resource folder in the data subfolder and can also automatically load the corresponding resource using the @Value annotation. This raw data is read in line by line and a list of documents is generated from it.
Document
org.springframework.ai.document.Document is a data container from the Spring AI framework, which is then used to fill the VectorStore or to read the data from it. In addition to the main text, any additional metadata can be stored in the Document, such as IDs, keywords or media such as images and videos. This metadata is saved together with the text and can later be used to filter or restrict the search results, thereby flexibly supplementing the semantic search. In our example, we deliberately do without metadata in order to keep the solution as lean as possible.
Vectorization
The actual vectorization or creation of the embeddings happens automatically when the store.add method is called. This task is also performed by the Spring AI framework, so that our code base can remain quite lean at the end of the entire implementation.
This is what our service looks like now:
Now we can start the service again and test it with the term “give me love songs”. To do this, we call up the following URL:
http://localhost:7070/similarity?text=give%20me%20love%20songs
The response should now look something like this:
[
{
"id": "f4b2ca2b-5822-4872-a5c9-7ca44c1d6096",
"text": "“All of Me” by John Legend is a romantic ballad about...",
"metadata": {
"distance": 0.4108433484775025
},
"score": 0.5891566515224975
},
{
"id": "c35320f2-8176-4e37-b46c-e69d58b8bfec",
"text": "“Love Lies” by Khalid & Normani is a romantic R&B song...",
"metadata": {
"distance": 0.4157826036156298
},
"score": 0.5842173963843702
},
...
]If you see this result, then congratulations, you have just implemented a semantic search!
Search results in detail
Let’s look at the results in detail to understand what we see there and how well the search actually works.
Here we see a list of documents. Each item is of type org.springframework.ai.document.Document, the default data type of VectorStore. By default, we get 4 results back, sorted in descending order of relevance compared to the search term.
"id": "f4b2ca2b...", - an unique id
"text": "“All of Me” by...", - original text of the found document
"metadata": {
"distance": 0.4108433484775025 - distance to the vector of the search term
},
"score": 0.5891566515224975 - relevance of the document foundThe id and text properties are almost self-explanatory, the interesting values are distance and score. Let’s take a closer look at these.
distance
SimpleVectorStore uses the so-called cousin similarity for the similarity comparison between the vectors. This method is one of the most common algorithms for comparing vectors. The cosine similarity measures the distance between two vectors and returns the result as a decimal number from 0 to 1. This result is also referred to as the distance. The shorter the distance, the closer the two vectors are to each other or the more similar the semantic meaning of the compared terms. A 0 means that both terms are identical, while a 1 means that the terms have practically nothing to do with each other.
score
Score stands for relevance, it is practically the same as the distance value, only reversed. 1 means an absolute match and 0 means no similarity at all. The score is simply calculated as 1- distance = score. This number is therefore always dependent on the distance. This number can be multiplied by 100, for example, and thus represent an easily understandable relevance in %.
Interpreting distance and relevance correctly
Caution is required when interpreting relevance in the vector search. Results often appear very appropriate, but show low relevance. This is often due to the fact that short search terms are compared with longer texts, especially in semantic searches.
Important points:
- Vectors of short and long terms are naturally different, which does not automatically mean lower relevance.
- Results with seemingly low relevance can also be important.
The relative ranking of the results is more meaningful than absolute distance values. - Normalization of the database is crucial, e.g. by adjusting the length of search terms and data records or structuring the data uniformly.
To summarize: The correct interpretation of relevance scores requires context and an understanding of the underlying vector comparisons.
Care should be taken when displaying relevance values in user interfaces, as raw values can be confusing. To avoid this, multipliers can be used to increase the values or alternative forms of presentation such as percentages or star ratings can be used.
Use alternative models
We are currently using the simple standard model all-MiniLM-L6-v2, which is contained directly in the package spring-ai-transformers-spring-boot-starter. When experimenting with different search terms, it is noticeable that some of the results are inappropriate. In particular, the search cannot currently process multilingual terms, e.g. in German. For example, “Queen” and “Königin” return completely different results, with the hits for “Königin” being barely relevant. The reason for this is that we use an English-only model. Therefore, we now include a more powerful, multilingual model and compare the results again.
We can configure the desired embeddings model in the application.yml by adding the following settings:
spring:
ai:
embedding:
transformer:
onnx:
model-uri: "https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2/resolve/main/onnx/model.onnx"
tokenizer:
uri: "https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2/resolve/main/tokenizer.json"model-uri: specifies the URL of the model to be used
tokenizer: uri: defines the appropriate tokenizer for the model.
These two components are essential for using a Transformer-based embedding model. The URL indicates that the models originate from the open source platform Huggingface.co, Huggingface is a kind of “GitHub for open source AI models”.
In principle, the model can come from various sources. It is important that it is a compatible Transformer ONNX model so that it can be easily integrated into the Spring AI environment.
Now restart the service and call up the following URLs one after the other:
http://localhost:7070/similarity?text=K%C3%B6nigin
http://localhost:7070/similarity?text=Queen
Both queries return similar results, which shows that the model not only understands English, but also German. Although our database currently consists exclusively of English terms, the model can process German search queries correctly, which is proof of how the multilingual embedding model works.
However, the relevance scores for German search queries are lower than for English search queries, as the language of the search query differs from the language of the database. However, the relevance remains consistent within the result set. Here we use a simple open source model that runs on a notebook and is less powerful due to its smaller vector dimensions. Nevertheless, we already achieve very useful results.
Choosing the right embedding model
As a general rule, smaller open source models often deliver mediocre results, especially with multilingualism. They work with fewer dimensions, but are easy on resources and run smoothly on notebooks or with limited hardware. Larger open source models offer better results, but require more computing power and are not always easy to use in corporate environments.
Alternatively, there are powerful commercial models such as OpenAI’s text-embedding-3-small or text-embedding-3-large, which are used via an API. These provide excellent semantic results, especially with multilingualism, and clearly outperform the simple open source models. The disadvantage: they are slower due to remote access, have API limits and incur costs, albeit often at reasonable prices. The decision ultimately depends on whether you work locally with limited quality, have sufficient hardware resources or prefer a premium model with outstanding semantic recognition.
Models with larger vector dimensions generate more data, which must also be stored accordingly in a vector database. This leads to a higher memory requirement, which should be taken into account when using such models.
Further expansion options
The basic functionality of our service is in place. However, there is still potential for optimization.
Keep VectorStore persistent
Currently, the master data is re-vectorized and loaded into the VectorStore every time the service is restarted. This can be made more efficient by persisting the state of the VectorStore in the file system. In this way, the already vectorized master data, i.e. our song data, can be loaded directly at the next start without being vectorized again, which saves time and resources.
All you have to do is call the save or load method of the SimpleVectorStore and specify the corresponding file path. In this way, the state of the VectorStore can be saved and loaded again directly at the next start.
@Value("classpath:data/songs.embeddings.json")
private val songsEmbeddingsRes: Resource? = null
@PostConstruct
fun init() {
(store as SimpleVectorStore).load(songsEmbeddingsRes!!);
}
fun saveVectorData() {
(store as SimpleVectorStore).save(File('/data/songs.embeddings.json'))
}If the file is stored in the resources folder of the Spring service, for example, it can be deployed directly to the target environment and even loaded into Docker or Kubernetes environments. When initializing the service, you can then load the persisted data from the embedded resources in a @PostConstruct method. This is particularly practical for smaller databases, as it eliminates the need for re-vectorization at startup and saves resources.
For larger projects or productive environments, you should instead use a specialized vector database such as: PostgreSQL/pgvector or ChromaDB. These are optimized for high-performance vector similarity processes and offer scalability and persistence for large amounts of data.
Caching embeddings models
In the standard Spring AI Transformers approach, embedding models are loaded from the source and cached locally the first time they are called. However, this means that the model and the tokenizer have to be downloaded again each time a new environment is deployed or a container is restarted. This leads to slower execution of the first request.
One optimization is to provide the models directly in the service so that they do not have to be reloaded every time they are started.
spring:
ai:
embedding:
transformer:
onnx:
model-uri: classpath:/model/model.onnx
tokenizer:
uri: classpath:/model/tokenizer.jsonThe corresponding model including tokenizer.json can be stored directly in the Resources Directory of the Spring Service and loaded via classpath:/model/path via application.yml.
Source code — GitHub
The complete source code for this article can be found in my GitHub repository. It also contains a small frontend with which the semantic search can be easily tested. In addition to text input, the search can also be tested using voice input via an integrated speech-to-text function.
Click here to go directly to the demo app
There is also a Docker image that can easily be started locally in a Docker environment so that the service is immediately ready for use.
docker run -p 7070:7070 ghcr.io/thecodemonkey/similaritysearch:mainEmbeddings use cases
A semantic search is just one of the possible examples that are possible thanks to word or context vectors. This technology is able to do much more because it is so fundamental. The ability to perform a semantic comparison mathematically opens up completely new possibilities, here is just a brief excerpt:
Recommendation systems - User and item embeddings are mapped in a common vector space to generate personalized recommendations. Similar users or products are identified via similarity search (e.g. “Customers who bought X also like Y”)
Automatic data labelling - Embeddings reduce manual annotations by automatically recognizing patterns and predicting missing labels (e.g. automatic categorization of support tickets)
Data quality improvement - By filtering noise and outliers, embeddings create more consistent data sets. They identify semantic inconsistencies, e.g. incorrectly assigned product categories
Clustering/classification - In knowledge graphs, embeddings group similar entities (e.g. “BVB Dortmund” and “FC Barcelona” as soccer clubs) and thus support entity matching or fraud detection
Knowledge graphs - embeddings can model relationships between entities (e.g. “Berlin → capital of → Germany”), which supports link prediction or knowledge discovery
Multimodal vector search (cross-modal retrieval) - Embeddings enable the transfer of texts and images into a common vector space. This allows search queries whether textual or visual to deliver cross-media results. For example: a search query such as “tropical beach at sunset” finds both photos of palm beaches and texts about travel destinations in Bali.
Best Practices
- Use ONNX models for maximum compatibility. ONNX models are platform-independent, standardized and can be easily used in different environments (local, cloud, various frameworks).
- Embeddings are model-bound. Note that embeddings created by a specific model are not compatible with embeddings from another model. Do not mix embeddings from different models within a vector database.
- Use the Simple Vector Store for small amounts of data: For quick prototypes or applications with a small volume of data, it is recommended to use the SimpleVectorStore from Spring AI. This stores vectors in the RAM and thus enables fast access times.
- Use of specialized vector databases for large amounts of data: In productive environments or with large data sets, specialized vector databases such as ChromaDB or pgvector Extension from PostgreSQL should be used to ensure efficient storage and retrieval of vectors.
- Note the amount of data for high-dimensional models: Models with many vector dimensions (e.g. 3000+) generate correspondingly large amounts of data, especially for long documents such as PDFs. Make sure that your vector database can store and process these volumes with high performance.
- Avoid redundant vectorizations: Vectors that have already been created should be saved and reused to avoid unnecessary calculations. Setting up suitable caches can increase efficiency here.
- Correct preparation of the data: Careful preparation of the data to be vectorized is essential. Uniform structures, such as the same length and similar structure of the texts, improve the quality of semantic recognition.
- Consideration of search term length: In semantic searches, search terms are often shorter than the entries in the database. This can lead to longer absolute distances, which do not necessarily reflect the actual relevance.
- Use of open source embeddings with limited hardware: If only limited hardware resources are available and the accuracy requirements are moderate, open source embeddings can be a viable solution.
- Select the appropriate model depending on the use case: Contextual embeddings are ideal for the semantic analysis of entire sentences, while models such as Word2Vec or FastText are better suited for the vectorization of domain-specific terms or abbreviations.
- Huggingface.co is a rabbit hole: The multitude of available open source models on Hugging Face can quickly become overwhelming and time-consuming. It is advisable to search specifically for suitable models for your own use case, e.g. via ChatGPT or Perplexity, and only then download the desired model from Hugging Face.
- Weighing up API-based premium models and on-premise solutions: Premium models such as OpenAI’s text-embeddings-3-xxx offer high accuracy, but can introduce latency and generate costs. On-premise open source models can handle significantly more embeddings requests, but may require compromises in accuracy.
- Optimize model deployment: To minimize first call delays, models and tokenizers should not be reloaded with each deployment. Local storage can help here.
Conclusion
Embeddings are a fundamental breakthrough in AI, as they make it possible to mathematically map semantic relationships between texts, images or even entire documents. This opens up completely new possibilities: from intelligent search and automatic categorization to personalized recommendations and complex knowledge management systems. For developers, this means that they can work with powerful AI techniques without needing in-depth expertise in machine learning, making access to modern AI applications considerably easier.
The integration of GenAI and embeddings in Spring is extremely simple and enables application developers in the enterprise environment to get started quickly without having to learn new technology stacks or programming languages. The Spring AI framework is mature and removes many complex tasks, allowing developers to stay in their familiar Java/Kotlin ecosystem and achieve productive results with just a few lines of code. Companies can rely on their existing teams as well as retain their grown, often complex system landscapes in the enterprise environment. Instead of completely replacing existing systems, AI functionalities can be integrated in a minimally invasive and iterative way, without a risky big bang migration. Spring AI thus builds a bridge between classic application development and modern AI!
Useful sources
Demo Application — https://simsearch.thecodemonkey.de
Sample Project — https://github.com/thecodemonkey/SimilaritySearch
Docker Image — docker pull ghcr.io/thecodemonkey/similaritysearch:main
Spring AI — https://docs.spring.io/spring-ai/reference/api/embeddings.html
Open Source Models — https://huggingface.co/
ONNX(Open Neural Network Exchange) — https://onnx.ai/
Transformer Architecture — https://huggingface.co/learn/nlp-course/chapter1/4
OpenAI#s Embeddings — https://platform.openai.com/docs/guides/embeddings
