Most ServiceNow AI projects I have been near start the same way. The licence is signed, Now Assist is switched on in sub-production, and a room full of people who know the platform well are suddenly using words like inference, grounding and model provider without a shared understanding of what any of them mean. The skills get enabled, the first summaries look fine, and the questions start when the security team asks where the data goes.
This is the introduction I wish every implementation team read first. No code, no configuration steps. Just the ideas and the words, in the order you will meet them, and the four risks every security conversation comes back to.
Four circles
Artificial intelligence is the big circle: any technique that gets a machine to do something that normally needs a human. Machine learning sits inside it: systems that learn patterns from data instead of following rules someone wrote. Deep learning sits inside that: machine learning done with neural networks many layers deep, which is what made it possible to learn from raw text and images rather than from tidy tables. Generative AI is the innermost circle: deep learning models that go beyond recognizing patterns to producing new content, text, images or code. Every LLM is a deep learning model. Not every deep learning model generates anything.
ServiceNow has had the second circle for years. Predictive Intelligence, which assigns incidents and suggests categories, is machine learning on structured data, rows and columns, the same family as a spam filter. What changed is the innermost circle. Now Assist and the AI agents are generative, built on deep learning models. They work on unstructured text, and they produce text back.
There is a fifth term that sits across the boundaries rather than inside one circle. Natural language understanding, NLU, is the branch concerned with machines reading human language. Virtual Agent has used it for years to work out what a user is asking for. Most modern NLU relies on machine learning, and generative models overlap with it whenever they are applied to text, which is why the dashed circle in the picture straddles the others.
That distinction matters on a project because the two behave differently. Traditional ML is predictable and narrow. Generative AI is flexible and occasionally wrong in ways that look right. Everything in the risk section below follows from that.
The words you will hear
Prompt, response, LLM, SLM, RAG, training and inference. If any of those is fuzzy, read Six Terms You Need Before Any Conversation About Generative AI first. It is vendor neutral and takes ten minutes. What follows is only how each of them shows up on ServiceNow.
The prompt in Now Assist is not what the user types. It is a template ServiceNow built, filled with record data such as the incident description and work notes, which is why a summary talks about your incident and not a generic one. The response is shown to a person who reviews it, and the platform labels AI-generated content so the reviewer knows what they are looking at.
ServiceNow uses both LLMs and SLMs. The Now LLM Service includes models ServiceNow trained itself, and the platform also calls third-party LLMs. ServiceNow’s own 12B model is an SLM built for Text-to-Flow, Text-to-Code, Text-to-Cypher and content moderation, tasks where a narrow, fast model beats a large general one.
RAG on ServiceNow retrieves records such as knowledge articles and puts them into the prompt. It respects access controls, so a user gets answers only from records they could open themselves, and it is the main reason answers stay current without retraining a model.
Training of Now LLMs happens in dedicated environments with strict access controls and never while customers are using the model. Inference runs on ServiceNow compute hubs, or on ServiceNow’s Azure public cloud endpoints for the OEM Azure OpenAI models, and prompts and responses are deleted after processing. Your prompts are not used to train anything.
The four risks and what the platform does about them
Every generative model shares four failure modes. The platform has a specific answer to each, and knowing the pairing is what turns a nervous security conversation into a short one.
| Risk | What it looks like | What ServiceNow does |
|---|---|---|
| Hallucination | A confident answer that is fabricated | Grounded prompt templates, RAG against trusted records, AI content labelled so a human reviews it |
| Adversarial attack | Hidden or malicious text in a record designed to hijack the model (prompt injection, jailbreaking) | Detection and blocking of injection attempts, skills designed to keep the attack surface small |
| Biased or corrupted training data | Poor quality training data producing poor quality answers | Models chosen on quality, Now LLMs fine-tuned on maintained fit-for-purpose datasets |
| Data leakage | A user prompting for data they are not allowed to see, or PII surfacing in output | Training data cleansed of PII and classified data, RAG bounded by access controls, AI Guardian blocking attempts to extract sensitive information |
The pattern in the right-hand column is worth noticing. None of the mitigations is “trust the model”. Each one puts a control around the model: what goes in, what comes out, who can see it. That is the mindset to carry into your own custom skills.
Where this goes next
That is the introduction. Everything above is true of any generative AI product, and the ServiceNow specifics so far are only the ones a whole project team needs on day one: what the prompt actually contains, who reviews the response, what RAG respects, and where training does not happen.
The implementation detail starts where this article stops. Which models sit behind which skills, how the Now LLM Service relates to the third-party providers you can now choose between, where provider choice is governed and where it is made, what a model card tells you and why you should read one for every model you use, and the short list of checks I run before the first skill is enabled. Each of those is its own article, and they follow this one in the ServiceNow AI category.
If you are joining a ServiceNow AI project this month, read this, read the six terms article, and you will follow every conversation in the room. The rest you can learn as the configuration happens.
