Language models (LLM) – how they work and what they are used for?

LLM language models (Large Language Models) are tools capable of generating and processing text, but in practice they do this by statistically predicting the next tokens based on context, rather than by “understanding” in the human sense. Their effectiveness is influenced, among other things, by the transformer architecture, the number of parameters and the way the text is split into tokens. These factors translate at the same time into answer quality, costs and processing speed (inference). If you want to choose a model consciously or set expectations realistically in a project, it is worth knowing what actually lies behind the parameters and tokenisation. In the following sections we discuss this step by step, without marketing simplifications.

What is an LLM language model and how do its parameters work?

LLM is a statistical model that creates text by predicting the next tokens based on context. It does not “understand” content in the human sense — it learns patterns from data and on that basis chooses the most probable continuations. The number of parameters (e.g. 7B, 70B) describes the scale of the network and usually is associated with better generalisation, but also with higher training and inference costs. In practice, opting for a larger model often improves quality across many tasks, however it requires more powerful hardware and a larger budget to run it.

Parameters do not operate in isolation, because the behaviour of an LLM is also influenced by the transformer architecture and the way it works with context. The self-attention mechanism determines which earlier tokens the model “looks at” when generating the next token, which makes it easier to connect distant parts of an utterance. At the same time, the model operates within a context window (e.g. 8k, 32k, 128k tokens), so it does not have “infinite memory” of the conversation. The larger the context and the larger the model, the easier it is usually to maintain coherence in longer tasks, but costs and latency increase.

1. 01Statistical modelPredicts the next tokens.
2. 02Scale of parametersGeneralisation vs. cost.
3. 03Architecture & contextSelf-attention mechanism.

Key point: an LLM is a statistical token predictor whose quality and cost increase with the scale of parameters and an effective attention architecture.

What is the significance of BPE tokenisation in language models?

BPE tokenisation (or SentencePiece) is very important, because an LLM does not work with words, only with tokens, which can be fragments of words, individual characters or parts of them. Tokens are the unit the model predicts, and they are also the basis for calculating processing costs in prompts and responses. In practice, 1,000 tokens usually equals about 700–800 words in Polish, but this depends on style and special characters. That is why the same “meaning” of an utterance may take a different number of tokens depending on how it is written.

Tokenisation also affects the model’s behaviour in technical content, because code, JSON and tables are simply sequences of tokens with characteristic syntactic patterns for an LLM. When text contains lots of symbols and structured layout (e.g. JSON), tokenisation can look different than in ordinary prose, which affects context length and inference cost. This is one of the reasons why in formal tasks people often keep an eye on the prompt length and the expected number of generated tokens. Sensible token planning helps keep the response within the context window and budget, especially in production applications.

The self-attention mechanism and its impact on context understanding

The self-attention mechanism means that the model “knows” which earlier parts of the text are most important when generating the next token. It works by calculating which earlier tokens should be given greater weight, allowing the model to connect distant elements of an utterance into one coherent response. In practice, self-attention makes it possible to refer back to definitions or assumptions from the beginning of a conversation, as long as they fit within the context window. However, this is not human understanding, but rather operating on statistical relationships learned from data.

The impact of self-attention on context also has its limits, because the model processes only as much content as is covered by the context window (e.g. 8k, 32k, 128k tokens). When the context becomes too long, the oldest information drops out or is compressed, which can create the impression of “forgetting” earlier assumptions. That is why in long conversations approaches such as intermediate summarisation, vector memory or selecting key fragments before the next question are used. As a result, answer quality depends not only on the model’s “intelligence”, but also on how you manage context in the application.

1. 01Token weightingCalculating the weight of fragments.
2. 02Connecting distant elementsLinking them into a coherent response.
3. 03Statistical relationshipsThis is not human understanding.
4. 04Limits of the context windowOlder information drops out.

Self-attention allows the model to focus on key relationships within the context window, but it is limited by its size and based on statistics.

Differences between GPT, BERT and T5 models

GPT, BERT and T5 differ primarily in their training approach and the tasks they are best suited to. Autoregressive models (e.g. GPT/Llama) predict the next token, so they naturally work well for text generation and conversation. BERT is a bidirectional model, which usually better supports tasks such as classification and information extraction from text. T5 has an encoder-decoder architecture, so it often performs well in “input → output” tasks, for example summarisation.

In practice, choosing the model family depends on whether you want to generate responses, or rather analyse text and pick out the important signals from it. If conversation and content creation are the priority, autoregressive models are the natural choice, because the way they generate output fits the “prompt → continuation” pattern well. When the task comes down to labelling content, categorisation or field extraction, a BERT-style approach is more likely to work well. If the goal is to transform text into a specific result (e.g. a summary with a clearly defined input and output), T5 can be convenient thanks to its encoder-decoder architecture.

What are the practical applications of LLMs in customer support automation?

LLMs in customer support automation are mainly used to classify tickets, suggest responses and ask for missing information, which genuinely shortens handling time. The model can immediately request the data needed to resolve the issue (e.g. an order number or logs), instead of carrying out a long exchange of initial messages. In a call centre environment, this is often combined with a base of articles and procedures so that the answers remain consistent with the current rules. When responses need to comply with procedures and up-to-date price lists, a typical approach is to connect the LLM to a knowledge base in RAG mode.

An LLM can also act as the “front layer” for company systems if it can call tools rather than guess the result. In practice, the model creates structured arguments (e.g. JSON), the backend executes an API request (e.g. CRM), and then the model presents the user with a concise summary of the result. This approach supports scenarios such as “check order status” and reduces the risk of misinformation, because the information comes directly from the source system. In production, it is also crucial to narrow the permissions of the integration (tokens with a limited scope) and audit actions, especially when operations are irreversible.

• Classification of tickets and assigning priorities based on the ticket content.
• Suggesting responses and clarifying questions about missing data (e.g. order number, logs).
• Answering based on company procedures and articles (RAG), rather than relying on the model’s “memory”.
• Calling tools and APIs (tool calling) to pull real data from CRM/ERP and summarise it for the customer.

1. 01Klasyfikacja zgłoszeńAutomatyczne sortowanie, szybszy przydział.
2. 02Sugerowanie odpowiedzi i dopytywanieKrótszy czas obsługi, natychmiastowe prośby o dane.
3. 03Wyszukiwanie w bazie wiedzy (RAG)Spójne procedury, aktualne cenniki i zasady.
4. 04Integracja z systemami (Tool Calling)Warstwa frontowa, ustrukturyzowane wyniki działania.

Summary: LLMs in customer support effectively support classification, speed up communication through suggesting and asking for clarification, ensure consistency thanks to RAG, and enable interaction with company systems through tool calling, shortening resolution time.

Data security in language models: how do you protect PII?

PII in systems with LLMs is protected, among other things, by masking personal data, controlling logs and retention periods, and configuring processing so as to limit the persistence of sensitive information. Users often paste data such as PESEL or addresses into chat, so the risk concerns not only the model’s responses, but also saved prompts and logs. In practice, log truncation, encryption and provider settings are used (e.g. a no-training-on-customer-data mode and limited retention). If the system does not have the right protection layers, PII can become embedded in logs or training processes, which is why safeguards need to be designed at the level of data, access and retention policy.

• Masking PII in input and output content and limiting logs to the absolute minimum necessary.
• Data encryption and limiting the retention and access to conversation records.
• Anonymisation of data used for training and “canary” tests to detect memorisation of sensitive sequences.
• Provider configurations that limit the use of customer data for training.

The risk of leakage rises especially when the model is fine-tuned on internal emails or tickets, and the data is too literal and not varied enough, which can cause the model to reproduce fragments in responses. That is why, in addition to anonymisation, access to training data is restricted and tests are carried out to check whether unwanted “memorisation” is taking place. It is also worth establishing clear rules within the organisation about what may be pasted into prompts, because users often unknowingly enter sensitive data into the system. Ultimately, PII security in LLM-based solutions depends on a combination of policies, technical safeguards and regular testing of the system’s behaviour.

Challenges related to hallucinations and how to limit them in LLMs

Hallucinations in LLMs are situations in which the model generates plausible-sounding but false information because it optimises for fluency and token probability, not truth. They are most effectively reduced when the system does not rely on the model’s “memory”, but instead supports responses with external sources or fact-checking. In practice, RAG with citations and requiring sources helps, making it clear which document fragments the answer was based on. In addition, you can reduce the randomness of generation to limit the tendency to fill in uncertain content.

Reducing hallucinations is also a matter of inference settings and validation on the application side, not just of a “better model”. When the response needs to remain stable and formal, a lower temperature, enforced formatting and output validation (e.g. JSON schema) usually help, and if there is an error, a correction loop too. If the task requires strict correctness (e.g. calculations or business rules), it is safer to delegate it to tools (calculator/Python, SQL, rules engine), leaving the model the role of interface and explanation. In document-based systems, it is also worth checking whether the answers are actually using the provided fragments, because simply “sticking” content into the prompt does not guarantee it will be used correctly.

How do API integration and access control affect LLM implementation?

API integration and access control determine whether an LLM can safely perform actions on company data rather than merely generating text. When the model has access to tools (tool calling), it can ask the system to perform a specific operation via the backend and then summarise the result for the user. This reduces “guessing” of facts, because key information is retrieved via API from source systems (e.g. CRM/ERP/Jira). At the same time, the broader the scope of tools and data, the more important security design becomes at the integration level.

Safe implementation is based on the principle of least privilege, activity auditing and consciously narrowing what the model can invoke at all. In practice, narrow-scope tokens, an allow-list of tools and “human-in-the-loop” for irreversible operations are used, and in sensitive areas also a “read-only” mode. This approach reduces the risk of abuse and mistakes, even if the model misinterprets a command or encounters a conflict of instructions in the context. In systems working with documents, prompt injection is an additional risk, so it is worth separating instructions from context and controlling which content fragments can influence decisions about tool use.