The question inside most procurement teams has already moved on. It is no longer whether officers are using AI — they are, often on their own initiative — but whether the organisation can say where it was used, what was checked, who remained accountable for the result, and where the information went.
The policy settings have moved with it. The Commonwealth’s updated policy for the responsible use of AI in government took effect in December 2025 and phases in across this year: documented plans for AI adoption, an internal register of use cases with a named owner against each one, impact assessments completed before deployment, and foundational AI training for every APS staff member. The first mandatory requirement landed in June; the remainder fall due in December. In Queensland, the QGEA Strategic AI Planning Policy has been mandated since September 2024, with agency governance arrangements expected to align to ISO/IEC 38507.
Procurement feels this earlier than most functions. It holds material that is commercially sensitive by definition — supplier pricing, unsuccessful responses, evaluation deliberations — and it produces a written record that has to stand on its own years later. Four decisions do most of the work, and one of them carries more weight than the rest.
1. Map the lifecycle before you choose a tool
Take the procurement lifecycle your agency actually runs — planning, market analysis, specification, approach to market, evaluation, negotiation, contract management, reporting — and mark each step one of three ways: suitable for AI, suitable with review, not suitable at all.
Most of the value sits early. Needs analysis, market scans, category research, first drafts, plain-English rewriting, and summarising long submissions are all tasks where a machine saves real hours without touching a decision. The clear stop is anything that scores, ranks or recommends a supplier. Evaluation is a judgement exercise attributed to named people, and a score an officer cannot explain in their own words will not survive a debrief, a complaint or an audit.
That map is not a preliminary — it is a governance artefact. It is what you show an auditor when they ask how AI is used in your function.
2. Draft with it, but make the review proportionate
Procurement runs on documents: procurement plans, evaluation plans, approach-to-market packs, briefing notes, supplier correspondence, contract summaries. First drafts are the fastest and safest win available.
The discipline is in the review, and the effort should scale with the consequence. A file note gets a read-through. An approach-to-market document gets checked line by line against the mandatory requirements, the approved budget and the policy that governs it. Four questions cover most of it: is it accurate, in its figures, thresholds and references; is it fair, or does the wording quietly favour an incumbent; is it proper, on probity grounds; and is it defensible, in that the officer whose name is on it can explain every clause.
Worth naming the specific hazard: AI drafts fluently, and fluency reads like authority. An error in a confident sentence is much harder to catch than an error in a clumsy one.
3. Set the guardrails, starting with where the data goes
Ask the question plainly. When an officer pastes a supplier’s pricing schedule into a chatbot to get a quick summary, where does that pricing schedule go? In most cases it leaves the agency’s environment, leaves the country, and comes to rest in a system governed by another jurisdiction’s law and another company’s retention settings. That is not a hypothetical risk to be managed later. It is a disclosure, and it takes about four seconds.
The rules already say where that material is allowed to live. Under the Australian Government’s Hosting Certification Framework, sensitive government data, whole-of-government systems and anything rated PROTECTED must be hosted on certified services — a free web tool is not certified hosting, and no individual officer holds the authority to make it so. In Queensland, disclosing personal information outside Australia is governed by section 33 of the Information Privacy Act 2009, and since the Queensland Privacy Principles commenced on 1 July 2025 state agencies have also operated under a mandatory notification of data breach scheme, with local governments joining on 1 July 2026. Under a notification regime, an unapproved paste stops being an internal embarrassment to be handled quietly and becomes a potentially notifiable event with a clock attached.
So before any tool touches information that is not already public, three questions need answers in writing. Where is the data processed and stored, and under whose law? Is anything entered into it used to train or improve the model, by default or by setting? Who inside the vendor can access it, for how long, and how is deletion evidenced? Sovereignty is not only a hosting location on a map — it is control over access, retention, and the ability to have your material returned or destroyed on your terms. A vendor who cannot answer those three questions contractually has answered the real question.
Data loss prevention has to see the new channel. Most agency DLP was designed and tuned for a world of email attachments, USB devices and file shares. Generative AI moved the exfiltration path to a text box in a browser tab, and a paste into a text box does not look like a file leaving the network. Close that gap deliberately: extend DLP and web filtering rules to cover AI services explicitly rather than by omission, tie enforcement to the classification labels already carried by procurement documents, log and monitor egress to those services, and prefer block-and-warn over silent blocking so officers learn where the boundary sits instead of simply hitting it.
Controls alone will not finish the job. Block everything and offer nothing, and the work migrates to a personal phone where no control sees it at all. Sovereignty and DLP only hold as a pair with a sanctioned, in-tenancy tool that officers can genuinely use for the tasks marked safe in decision one. Give people an approved path and enforcement becomes reasonable; give them a wall and it becomes theatre.
Two guardrails complete the set. Approvals and records: if AI helped produce a document that supports a decision, the file should say so, and the use case should appear on the register with a named owner against it. And suppliers, which is the boundary most agencies have not yet drawn. Your suppliers are using AI inside services you have already bought, and their sovereignty posture becomes yours the moment your data reaches them. The Digital Transformation Agency’s model AI clauses are optional and meant to be tailored, but they set a useful benchmark: written approval before a supplier uses an AI system to deliver, quality assurance on its outputs, records of what was used, and prohibited systems excluded outright. Add the residency questions to that list. If they are not being asked, you do not know what is operating in your supply chain.
4. Make it repeatable, and keep it light
The teams getting real value do the unglamorous part. A small set of tested prompts for the tasks they repeat every month. A one-page review checklist that sits beside the evaluation plan. A two-line rule, written in plain words, on what may never be entered into an unapproved tool. Three or four AI and data residency questions added to the standard supplier questionnaire. A line in the delegation that names what AI is not used for. Then a quarterly look at whether any of it still fits.
A twelve-page AI policy nobody reads is worse than a one-page rule everybody follows. Most of the obligations here are not new — probity, value for money, record-keeping and a decision a named officer can defend are the same obligations that governed the last twenty years of public procurement. What AI adds is volume moving through those controls at speed, and one genuinely new question that every agency has to answer for itself: does the information stay where the law, the framework and the contract say it must?
GovP2P works with Australian government departments and government owned corporations on procurement that is compliant, auditable and defensible. If your team is working out where AI fits, where it should not, and how to keep agency information inside the boundary — book a discussion with us.
