How software teams turn experience into shared, open and controllable knowledge infrastructure

The first phase of generative Artificial Intelligence adoption in software development focused heavily on prompt engineering. The basic assumption was that a skilled user, by formulating a request correctly, could obtain better code, more complete documentation or more accurate technical analysis.

This assumption was not wrong, but it was incomplete. As language models and coding agents take on increasingly complex tasks, it becomes clear that output quality does not primarily depend on a clever formulation. It depends on the context provided to the model, the quality of the available sources, the state of the repository, the verification mechanisms in place and the engineer’s ability to identify what is missing from an apparently correct solution.

A team’s real advantage therefore does not lie in a library of prompts or in access to the most powerful commercial model. It lies in its ability to transform experience, technical decisions, failures and quality criteria into shared, reusable and auditable knowledge infrastructure.

The critical question is not only which model produces better code. It is also who controls the data, the context, the organisational memory, the evaluation mechanisms, the cost and the ability to replace the model.

A good question is compressed experience

An experienced engineer rarely asks only, “How should this feature be implemented?” Instead, they examine which assumption may be wrong, which invariant may be violated, which dependency should not be introduced, how the change will behave under load and which test will demonstrate that the solution is actually correct.

Even a brief observation such as “Does this conversion create an unnecessary copy of the data?” may reflect years of experience in memory management, algorithmic complexity and efficient system design.

An experienced developer’s question incorporates the history of the system, its architectural assumptions, accumulated technical debt, previous failures and actual acceptance criteria. In this sense, effective prompts are traces of tacit knowledge. They are not, however, organisational knowledge by themselves.

A prompt library stores formulations, but not necessarily the reasoning that made them effective. When context, constraints, failed approaches and verification methods are missing, a prompt can easily become a recipe applied mechanically to the wrong problem.

For knowledge to become useful to the entire team, the organisation must preserve not only what was requested, but also why it was requested.

Coding agents increase the value of expertise

A representative example comes from a real change to the open source uutils project. Diomidis Spinellis used a coding agent to modify how the project’s implementation of the sed command handles characters and raw bytes.

The change affected 13 files and involved 1,740 additions and 609 deletions. The agent carried out much of the laborious work, while the code remained compilable and executable throughout the process. Unit tests, integration tests and continuous integration checks played a decisive role.

However, 61 of the 78 prompts and individual instructions, or 78 per cent, concerned improvements identified and requested by the experienced engineer. These observations addressed architectural choices, unnecessary data conversions, algorithmic complexity, the reuse of existing functions, edge cases, naming, documentation and missing tests.

The conclusion is not that AI agents are ineffective. On the contrary, they can significantly accelerate development when they operate within a mature technical environment.

Producing code, however, is not the same as producing reliable software. Whether code compiles can be checked automatically. Determining whether a function has quadratic complexity, whether a conversion changes data semantics or whether a new abstraction violates the conventions of the project requires deeper knowledge.

The agent accelerates execution. The expert still defines what a correct result means.

From prompts to context engineering

A mature organisation does not simply store conversations with models. It transforms useful human-machine interactions into structured knowledge assets.

For every significant task, the team should be able to retrieve:

• the description of the problem
• the operational or business rationale
• the technical constraints
• the relevant architectural decisions
• the alternatives considered
• the success criteria
• the known edge cases
• the associated tests

This knowledge can be represented through architecture decision records, runbooks, issue templates, postmortems, checklists and test suites.

A Retrieval-Augmented Generation architecture can then retrieve the appropriate material for each task. The model should not be treated as the repository of organisational truth. The source of truth remains the code, technical documentation, data schemas, approved documents and test results.

This is where the transition from prompt engineering to context engineering takes place. The important question is no longer only what should be written in the prompt. It is also:

• which files the agent can access
• which tools it can execute
• which data it is authorised to process
• which version of the documentation is valid
• which tests must pass before the result is accepted
• which model is appropriate for the task

The prompt is merely the visible entry point into a much larger system.

Tests are executable organisational knowledge

Documentation describes what should happen. Automated tests verify whether it actually happens.

Every unit, integration or regression test may encode a functional requirement, an invariant, a previous failure, an edge case or a security requirement. When an agent works in a repository with extensive test coverage, it is guided not only by the natural-language prompt. It is also guided by an executable set of rules defining acceptable behaviour.

Code produced with AI assistance should pass through:

1. compilation or type checking
2. formatting and linting
3. static analysis
4. dependency checks
5. unit and integration tests
6. regression testing
7. security scanning
8. performance testing where necessary
9. human code review

Every significant error should return to the system as a new test, a new rule, an update to the documentation or a new architectural decision.

In this way, a failure does more than correct a specific change. It increases the future capability of the entire team.

A controlled development loop

The use of a coding agent can be organised as a controlled process:

Issue → context bundle → agent execution → automated checks → human review → knowledge update

The context bundle may include the relevant issue, affected files, architecture decision records, API contracts, tests, security instructions and acceptance criteria.

The agent performs the task in an isolated environment and produces code, tests and documentation. Automated checks are then executed. The engineer reviews the diff, the assumptions, the architectural effects and the test results.

Once the change has been accepted, the useful conclusions return to the knowledge base. A test may be added, a runbook may be updated or a new architecture decision record may be written.

This feedback loop is more important than the prompt itself.

Evaluation becomes the critical skill

As code generation becomes faster, code evaluation becomes more important and often more difficult.

A junior developer can now produce more code than they are capable of reviewing. This creates a serious educational problem. If initial implementations are repeatedly delegated to agents, how will less experienced engineers learn to recognise:

• incorrect abstractions
• algorithmic complexity problems
• hidden dependencies
• security vulnerabilities
• inconsistent contracts
• difficult-to-maintain code
• solutions that work only in the obvious case

Organisations need structured apprenticeship, systematic code review, collaboration between experienced and junior engineers and the study of real failures.

AI can accelerate learning when the developer is required to explain the solution, examine alternatives, write or review the tests and defend the technical choices.

When the user simply accepts the generated result, the organisation risks skill atrophy and dependence on code that nobody fully understands.

Preserve the “why”

The most useful information from an agent session is often not the final code, but the route that led to it.

Which assumption was challenged? Which information was missing? Which alternative was rejected? Which test exposed the problem? Which piece of evidence changed the decision?

After a significant task has been completed, the engineer and the system can produce a concise learning record containing:

• the final decision
• the relevant context
• the critical assumptions
• the problems identified
• the tests added
• the lessons that can be reused

It is not useful to store every conversation automatically. Transcripts contain noise, provisional assumptions, incorrect paths and potentially confidential information.

Organisational memory should consist of curated, versioned and approved knowledge assets, not an uncontrolled accumulation of conversations.

Open infrastructure and control over knowledge

Prompts sent to an external model may reveal the architecture of a system, technical weaknesses, operational priorities and internal decision-making processes. Organisational knowledge does not exist only in documents and code. It also exists in the questions employees ask.

For this reason, the choice of AI infrastructure is a matter of security and knowledge governance.

Closed cloud services provide rapid access to powerful models, but they may create dependency in relation to:

• APIs and interfaces
• pricing
• usage limits
• data retention policies
• security controls
• the continued availability of specific models

Locally or privately hosted open models allow the organisation to keep code, documents, architectural records and internal interactions within infrastructure it controls.

This does not mean that every organisation must reject all external services. It means that data and tasks should be classified and routed deliberately. Sensitive information and critical operations can be processed locally, while external models may be used selectively for low-risk or unusually demanding tasks.

Fully open models and open-weight models

Not all models described as “open” provide the same level of openness.

A fully open model publishes, to the greatest practical extent:

• model weights
• training code
• data preparation methods
• checkpoints
• evaluation results
• documentation of the development process

An open-weight model can be executed and adapted locally, but it may not provide sufficient information to reproduce or fully audit the process through which it was created.

This distinction has practical importance. For a low-risk internal application, an efficient open-weight model with an appropriate licence may be sufficient. For systems affecting critical operations, public services or individual rights, the requirement for technical transparency and independent evaluation becomes stronger.

Model selection should therefore consider:

• the licence
• commercial-use conditions
• provenance and documentation
• performance on the organisation’s actual tasks
• computational requirements
• the ability to run locally
• the level of application risk
• the ability to replace the model

Use the smallest sufficient model

An efficient AI system does not send every request to the largest available model. It uses model routing and selects the smallest model capable of completing the task reliably.

Smaller models can handle:

• document classification
• metadata extraction
• named-entity recognition
• request routing
• simple summarisation
• embedding generation
• basic text transformation

More powerful models can be used selectively for complex analysis, demanding programming tasks, architectural design or the evaluation of alternative solutions.

A shared API gateway allows applications to call different models through a stable interface. The model becomes a replaceable component rather than the foundation on which the entire application is locked.

A hybrid architecture may combine:

• small local models for frequent tasks
• more powerful local models for sensitive workloads
• external models for selected complex tasks
• RAG for access to approved organisational knowledge
• human validation for critical decisions

Open infrastructure means replaceability

The advantage of open infrastructure is not limited to the model itself. It lies in the ability to build a modular stack that may include:

• an open operating system
• an inference server
• an API gateway
• identity and access management
• search infrastructure
• a vector database
• a metadata store
• monitoring and observability tools
• audit logging
• evaluation tools
• cost and usage dashboards

The critical property is that every layer can be replaced independently.

An organisation can change the model without rewriting the application, migrate the data to a different environment, replace the inference engine or apply common security policies across multiple models.

When the model, interface, data, workflows and pricing are controlled by the same provider, changing the solution becomes technically and economically difficult.

Collective knowledge requires governance

The technical stack is not sufficient. The organisation must know:

• which models are being used
• which tasks they are approved for
• which data they may process
• who has access
• how critical actions are logged
• which evaluation sets are used
• who approves production deployment
• when a model should be upgraded or withdrawn

A basic governance framework may include:

• a model registry
• model cards
• dataset documentation
• evaluation sets based on real organisational tasks
• versioning of prompts and context policies
• audit logging
• an incident and error-reporting process
• a model deactivation procedure
• rules for human oversight

Outputs that affect rights, financial decisions, security or critical technical operations should not rely only on the internal parameters of the model. They should be grounded in approved sources, accompanied by evidence and validated by an accountable human.

At the same time, capturing employee knowledge must not become a mechanism for surveillance or the appropriation of individual contributions. Clear rules, attribution, correction rights and transparency about reuse are essential.

Cost must be measured together with value

AI agents are not free. Beyond licence fees or computing resources, they require experienced engineers to prepare context, guide execution, review changes, correct errors and maintain the supporting infrastructure.

Complex sessions may consume large numbers of tokens, particularly when an agent repeatedly reads a large repository, runs tools and revises its solution.

Local models are not automatically free either. They require hardware, electricity, monitoring, upgrades and technical support. They can, however, provide greater cost predictability for stable and repetitive workloads.

The correct question is not simply whether a task can be performed using AI. It is:

• whether the result reaches the required quality
• how much engineering time it saves
• how much review time it requires
• how many iterations are needed
• how many defects it introduces
• what the long-term maintenance cost will be
• what the smallest sufficient infrastructure is

Lines of code and initial generation speed are not reliable productivity metrics. More useful measures include review time, acceptance rates, defects identified before and after integration, rollback frequency, cost per successful task and impact on maintainability.

The new competitive advantage

As prompting techniques become common knowledge, competitive advantage moves towards the organisational system surrounding the model.

The teams that benefit most are those with:

• clear and current documentation
• reliable sources of truth
• extensive automated testing
• mature development platforms
• fast feedback loops
• well-defined access policies
• measurable quality criteria
• a culture of shared learning

AI agents do not eliminate the need for experts. They increase the value of those who can define the problem correctly, select the appropriate context, design the verification process and identify failures that are not visible at the level of syntax.

The meaningful outcome is not simply more code. It is the creation of a system in which every task, every review and every failure improves the collective capability of the team.

In this context, local open models and an open technology stack are not only technical or economic choices. They are the mechanisms through which knowledge, data and operational rules remain controllable assets of the organisation rather than becoming dependencies on an external platform.

Source of this article: blog.glossapi.gr