Artificial Intelligence Basics

Artificial intelligence (AI) is the broad field of building computer systems that perform tasks normally associated with human intelligence, such as understanding language, recognizing patterns, making predictions, generating content, and selecting actions.

For software engineers, the important distinction is that AI systems are usually probabilistic rather than explicitly programmed. Traditional software follows rules written by developers; modern AI often learns behavior from data and produces outputs based on statistical patterns. This changes how applications are designed, tested, monitored, scaled, and secured.

Table of Contents

• What Artificial Intelligence Means
• How AI Systems Work
  • Training
  • Inference
• Major Types of AI Systems
  • Predictive AI
  • Generative AI
  • Decision-Making AI
• AI Models and Applications
• Why AI Is Probabilistic
• Building AI into Software Systems
  • Treat AI as an Unreliable Dependency
  • Validate AI Output
• Production Trade-Offs
• Conclusion
  • More Articles to Read

What Artificial Intelligence Means

AI is an umbrella term rather than one specific technology. Machine learning, neural networks, deep learning, large language models, computer vision, recommendation systems, and autonomous agents can all be parts of AI.

A useful engineering definition is: an AI system uses a model to transform input into a prediction, generated output, classification, score, or action.

Consider a payment system deciding whether a transaction might be fraudulent. A traditional implementation could contain manually written rules:

def is_suspicious(amount: float, country_changed: bool) -> bool:
    if amount > 5000:
        return True

    if country_changed and amount > 1000:
        return True

    return False

The behavior is deterministic. Given the same values, the function always follows the same explicitly defined conditions.

A machine-learning system can instead receive features describing the transaction and return a probability:

features = {
    "amount": 2400,
    "transactions_last_hour": 8,
    "distance_from_previous_transaction_km": 1800,
    "account_age_days": 42,
    "merchant_risk_score": 0.71,
}

fraud_probability = model.predict(features)

if fraud_probability > 0.90:
    require_manual_review()

The developer defines the surrounding application logic, but the relationship between those features and the fraud probability was learned from data rather than encoded as a sequence of business rules.

This distinction is fundamental to understanding modern AI applications.

How AI Systems Work

Most modern AI systems can be understood through two different phases: training and inference. Training creates or adjusts the model; inference uses that model to process new input.

Training

During training, an algorithm processes data and adjusts internal parameters so that the model becomes better at a target task. Depending on the model, this can mean learning to classify images, estimate delivery times, predict customer behavior, or generate text.

Suppose historical logistics data contains millions of completed deliveries:

Historical deliveries → Training process → Delivery-time model

Each record might contain distance, carrier, warehouse, destination, package type, weather conditions, dispatch time, and actual delivery duration. Training discovers statistical relationships between those inputs and observed outcomes.

Modern neural networks can contain millions or billions of adjustable parameters. Training repeatedly changes these values to reduce an objective called a loss function. The resulting parameters encode patterns learned from the training data.

This process can be computationally expensive. Large models may require distributed GPU clusters, large datasets, checkpoint storage, and significant training time. Application developers, however, often consume models that have already been trained instead of training them from scratch.

Inference

Inference happens when a trained model processes new input and generates an output.

A delivery-time model might receive:

{
  "origin": "Dallas",
  "destination": "Austin",
  "distance_km": 315,
  "carrier": "carrier_a",
  "package_type": "parcel"
}

and produce:

{
  "estimated_delivery_hours": 21.4,
  "confidence": 0.87
}

Inference is the phase most backend engineers encounter when integrating AI into applications. The model may run inside the application's infrastructure or behind an external API.

Training performance and inference performance are separate engineering problems. A model that is practical to train might still be too slow, expensive, or memory-intensive to serve thousands of requests per second.

Major Types of AI Systems

AI systems can be categorized in many ways, but application engineers frequently encounter three practical behaviors: systems that predict, systems that generate, and systems that select actions. Real applications often combine all three.

Predictive AI

Predictive systems estimate something about an input or future event. Common examples include fraud detection, recommendation ranking, demand forecasting, anomaly detection, spam filtering, churn prediction, and delivery-time estimation.

A model may return a class:

{
  "classification": "fraud"
}

or a numerical score:

{
  "fraud_probability": 0.94
}

Scores are often more useful in production because business logic can decide how different confidence levels should be handled. For example, a high-risk transaction can be rejected, a medium-risk transaction sent for review, and a low-risk transaction accepted.

Generative AI

Generative AI creates new content based on patterns learned during training. The output may be text, source code, images, audio, video, structured data, or other representations.

Large language models are a prominent example. An application sends text or other supported input to a model, and the model generates a sequence of output tokens.

Typical applications include customer-support assistants, document extraction, summarization, code generation, search assistants, translation, and natural-language interfaces over existing systems.

Generative systems introduce a particularly important engineering property: plausible output is not necessarily correct output. A fluent response can contain incorrect facts, invalid identifiers, nonexistent APIs, or unsupported conclusions. This requires validation and application-level controls.

Decision-Making AI

Some AI systems use model output to select an action rather than simply return information. Examples include routing a support ticket, choosing a recommendation, selecting a warehouse, invoking a software tool, or deciding which step an AI agent should perform next.

The consequences are different from ordinary generation. An incorrect generated sentence may be inconvenient; an incorrect action could modify data, send a payment, expose information, or trigger an expensive external operation.

The more authority an AI system receives, the stronger its validation, authorization, audit, and failure-containment boundaries must become.

AI Models and Applications

An AI model is usually only one component of a production AI application. The model performs inference, while conventional software handles authentication, authorization, data retrieval, validation, persistence, caching, rate limiting, observability, and business rules.

A document assistant, for example, might contain an API service, authentication layer, document store, embedding model, vector database, retrieval service, large language model, and response-validation logic. Only some of those components are AI models.

This distinction prevents a common architectural mistake: treating the model as the application.

AI therefore does not replace conventional system design. It introduces another dependency with unusual latency, cost, correctness, and security characteristics.

Why AI Is Probabilistic

Traditional software is usually expected to produce deterministic results. Calling a tax calculation function twice with identical inputs should normally produce identical outputs.

Many AI systems behave differently. Their outputs represent learned statistical relationships, probabilities, or generated sequences. Two inputs that appear almost identical can sometimes produce meaningfully different outputs.

Generative models can introduce additional variation during decoding. Parameters such as temperature and sampling strategy affect how the next output token is selected.

This changes the meaning of correctness testing. A test such as:

assert summarize(document) == expected_summary

is often too strict for a generative system because several different summaries may all be acceptable.

Evaluation may instead examine properties such as factual correctness, relevance, completeness, groundedness, format validity, safety, or whether required information appears in the result.

Probabilistic behavior does not mean that application behavior itself must become uncontrolled. Deterministic software should constrain probabilistic model output wherever correctness matters.

For example, an AI system may suggest a shipping service, but application code should verify that the returned carrier and service identifiers actually exist before creating a shipment.

Building AI into Software Systems

Production AI integration is primarily a systems problem. The model has to operate inside latency budgets, availability requirements, security boundaries, cost limits, and existing business workflows.

A backend service calling an external model might look conceptually similar to any other remote dependency:

from dataclasses import dataclass


@dataclass
class Classification:
    category: str
    confidence: float


def classify_ticket(text: str) -> Classification:
    response = ai_client.classify(
        text=text,
        timeout_seconds=3,
    )

    return Classification(
        category=response.category,
        confidence=response.confidence,
    )

The important engineering work starts around this call: deciding what happens when inference takes ten seconds, the provider returns an error, the result violates the expected schema, confidence is low, or request volume suddenly increases.

Treat AI as an Unreliable Dependency

Model inference can fail because of provider outages, rate limits, network errors, overloaded GPU capacity, invalid requests, safety filters, or internal model-serving failures. Applications should define explicit timeouts and failure behavior instead of allowing model calls to block indefinitely.

Retries require care. Retrying every failed inference immediately can amplify an outage and increase cost. Retries should normally be bounded and reserved for failures that are actually transient.

Fallback behavior depends on the business operation. A support-ticket classifier might fall back to a default queue. A search assistant might return ordinary search results. A low-confidence financial decision might require human review rather than another model guess.

The correct fallback is therefore a product and reliability decision, not merely an AI configuration.

Validate AI Output

Generated output should not automatically become trusted application state. When structured data is expected, enforce a schema and then validate business constraints independently.

ALLOWED_PRIORITIES = {"low", "normal", "high", "urgent"}


def validate_priority(value: str) -> str:
    if value not in ALLOWED_PRIORITIES:
        raise ValueError("Model returned an unsupported priority")

    return value

Schema validation proves that output has the expected shape. It does not prove that the output is factually correct or permitted by business rules.

The same principle becomes more important when models can invoke tools or perform actions. Model output should be treated as a proposal that passes through normal authorization and validation boundaries before execution.

Production Trade-Offs

AI introduces several constraints that are less prominent in conventional request-response services. Model quality is important, but production architecture also has to balance latency, throughput, cost, reliability, and operational control.

Monitoring should reflect these characteristics. Useful signals include p50/p95/p99 inference latency, request rate, timeout rate, model errors, token consumption, cost per request, output-validation failures, fallback rate, and model-specific quality metrics.

Capacity planning also differs from ordinary CPU-based APIs. Large models can be constrained by accelerator memory, memory bandwidth, batch size, context length, and token-generation speed. Scaling from ten to ten thousand concurrent AI requests may therefore require architectural changes rather than simply adding generic application servers.

Another common mistake is using AI where deterministic software is better. A model is unnecessary for exact arithmetic, straightforward validation, fixed business rules, or database lookups with known semantics. Use probabilistic computation when the problem benefits from learned patterns or flexible interpretation; keep deterministic operations deterministic.

As AI applications become more complex, these concerns expand into dedicated architecture patterns involving model gateways, retrieval systems, vector databases, queues, caches, evaluation pipelines, and observability. AI Application Architecture covers those system-level decisions in more depth.

Conclusion

Artificial intelligence is best understood as a collection of techniques that allow software to make predictions, generate outputs, or select actions using patterns learned from data. Modern AI systems typically separate expensive model training from inference performed by production applications.

For software engineers, the key architectural change is uncertainty. Model output can be slow, expensive, probabilistic, or incorrect, so AI should operate inside deterministic application boundaries that provide validation, authorization, failure handling, observability, and business rules. The model is an important component, but the surrounding software system determines whether AI can operate safely and reliably in production.

More Articles to Read

• Machine Learning Basics
• Neural Networks and Deep Learning
• Large Language Models (LLMs)
• AI Application Architecture