If an artificial intelligence can plan and act on its own, who answers for its actions? That is the question at the heart of the webinar “Agentic AI: tra autonomia e responsabilità” (Agentic AI: between autonomy and accountability), organised by AgID Academy, the training arm of Italy’s national digital agency, on 13 April 2026.
The discussion, moderated by Paola Liberace, brought together Giuseppe Italiano, Vice-Rector for AI at LUISS Guido Carli, and Mario De Caro, holder of the UNESCO Chair in Ethics of AI at Roma Tre University. They were joined by Mario Nobile, Director General of AgID.
Three different perspectives, technical, ethical and institutional, reaching a shared conclusion: machine autonomy must come with clear rules on who decides and who is accountable. Here we go through the main points and what they mean for those who, like us, bring AI into companies and healthcare.
From AI agent to agentic AI#
Giuseppe Italiano starts from a distinction that often gets lost in public debate. An AI agent uses a language model as its engine, but it does not stop at conversation: it acts on external resources such as databases, search engines and digital platforms.
Three elements define it: a scope of action set by whoever assigns its goals, an interface for talking to users, and access to specific tools. Italiano compares it to a digital employee to whom we hand a virtual desk to get a piece of paperwork done.
Agentic AI is something else: it is the organisation of work, that is, the way several agents are coordinated to reach a goal. Coding assistants are the example. They write a first draft of the code, analyse it, run it in a protected environment, fix the errors and start again until the result works.
This leap brings three new challenges:
- Certification. A single agent can be certified like a product. An agentic system requires certifying processes and behaviours that change over time.
- Distributed responsibility. When several agents work together, it becomes hard to establish who performed an action, why, and with what effects.
- Governance. Someone has to decide what each agent does, who coordinates them and to whom each single step is attributed.
In short, we move from managing information to managing processes. And the better agents get at moving on their own, the harder it is to follow the path that leads them to a conclusion.
Italiano does not treat this as a cause for alarm. We used fire long before we understood its chemistry, and the steam engine before thermodynamics. But he points out that even a proactive public service, such as a State that acts on its own initiative when a child is born, takes initiatives, uses personal data and coordinates several bodies. That is why clear rules, transparency and responsibilities are needed.
Moral agents without rights#
Mario De Caro moves the discussion to ethics. A moral agent is an autonomous entity whose decisions can do good or harm to others.
Throughout history the moral sphere has widened several times. First slaves, then women, who voted for the first time only in 1893, in New Zealand. Today we also recognise duties towards animals, children and future generations. These are entities that have rights but no duties: in philosophical terms, they are “moral patients” but not agents.
With agentic AI the opposite happens for the first time. These systems act and affect people’s lives, so they must respect moral constraints. But they have no rights, because they have no consciousness and cannot suffer. On the possibility of a future artificial consciousness, De Caro notes, experts are split down the middle.
Why are constraints needed? Because these systems are autonomous, hard to predict and hard to explain. They are not deterministic like the old AI, but probabilistic. The extreme case De Caro cites is autonomous military drones: a system with a single objective and no limit on what it may hit to achieve it.
Right decisions for the wrong reasons#
The most concrete point in De Caro’s talk concerns explainability. A system can often give the right answer without our knowing whether it gets there for the right reasons.
The example has become a classic. A model trained to tell wolves from huskies was right 95% of the time. Analysing its errors revealed that the wolf photos almost always had snow in the background. The system was not recognising the animal: it was recognising the snow.
With wolves and huskies the mistake is harmless. In medicine it is not. De Caro cites a recent study according to which, for rare diseases, AI systems get the diagnosis right far more often than physicians. But if a model bases its diagnosis on a collateral symptom, perhaps due to another cause, sooner or later it will get it wrong without anyone noticing in time.
Hence the question running through the whole webinar: when an autonomous system gets it wrong, whose responsibility is it? The developer’s, the adopter’s, the user’s or the institution’s?
The lesson of autonomous driving#
Mario Nobile answers with a by now familiar comparison: the levels of autonomous driving, from 1 to 5. The self-driving car is perhaps the AI agent par excellence.
His starting point is provocative. In Italy around 3,000 people die on the roads every year. Leaving everything to the human being, with no tools to prevent driving while impaired, has a cost. And it does not even guarantee accountability: without black boxes on board, mandatory in other markets, it is often impossible to establish whether an accident was due to human error or a fault.
The same applies to public administration. When a benefit or a social housing unit goes to the wrong person, the blame lies clearly with neither the official nor the algorithm. But someone has to answer for it.
Nobile’s position is clear-cut:
- Up to level 4, yes. The machine helps, reduces errors and harm, but the human being retains responsibility. The steering wheel and the gearbox stay where they are.
- Level 5 is a long way off. Total autonomy means entrusting the machine with moral choices on which human beings themselves do not agree.
Moral Machine, the MIT project on the trolley problem, shows this. If a car with no brakes has to choose whom to run over, the answers change with culture, religion and personal involvement. How can we delegate to a machine what we have not managed to define among ourselves?
What it means for those who build AI#
For those who develop AI systems every day, the message of the webinar translates into concrete design choices. They are the same ones that guide our work.
- The human being stays at the wheel. KOI supports anaesthetists in the ASA-PS classification of patients, but the final decision rests with the physician. It is the “level 4” Nobile talks about: the machine reduces the errors, the person answers for them.
- Every action must be traceable. In TIDE’s agents, as in any agentic system, it must be possible to reconstruct who did what and why. Without traceability, distributed responsibility becomes nobody’s responsibility.
- Data stays under control. An agent that uses personal data and coordinates several systems must do so within a defined perimeter. That is why our PRISMA infrastructure runs on-premise or on European cloud, in line with the GDPR and the AI Act.
- Critical cases go through a person. When a system automates a workflow, anomalies must be flagged for human review, not resolved silently.
Agentic AI can make organisations and public administrations more efficient and, as Italiano said, even more present in people’s lives. But autonomy only works if it is clear, beforehand, who decides and who is accountable.