About the position
Context
In the context of the ANR FM2AI, and in collaboration with APHP, the aim of this internship is to systematically evaluate LLM-based agents that combine reasoning with specialized medical imaging tools for abdominal CT question answering, focusing on their accuracy, computational efficiency, the contribution of individual tools, and the faithfulness of their reasoning.
Scientific context:
Foundation models for CT have multiplied: vision‐language encoders (CT‐CLIP [10], Merlin [8], Jolia [15]), generative models that answer questions about a volume (CT‐CHAT [10], MedGemma 1.5 [11]), large pretrained encoders (Pillar‐0 [12], Curia [13], FlexiCT [14]) and segmentation tools (TotalSegmentator [9]). On their own, they still fail on tasks that require localizing, measuring and then reasoning.
Hence the idea, following ReAct [16], of delegating the reasoning to an LLM that calls these models as tools. Born in chest radiography with MedRAX [7], this paradigm now reaches 3D CT: CT‐Agent [2], 3DMedAgent [5], CT‐Flow [6], INFORM‐CT [3].
The most striking result comes from RadAgent [1]: without any training, an agent orchestrating CT‐CHAT, CT‐CLIP, TotalSeg‐mentator and a general‐purpose 2D VLM (Gemma 3 [19]) outperforms CT‐CHAT on report generation. On DeepTumorVQA [4], a fine‐tuned 2D VLM even beats the best 3D model (66.3% vs. 59.8%), but without measurement tools, all models make up quantitative values.
The problem
: each system comes with its own harness (agent loop, tools, prompts), data and metrics, so it is impossible to tell what actually makes the difference. Abdominal CT, less studied than the chest, is an ideal testbed: DeepTumorVQA pro‐vides questions with verifiable answers on 9,262 public volumes, organized by level (recognition, measurement, reasoning), together with a standardized tool environment.
Assignment
Research questions:
1. What makes a good agent, and at what cost? Compare harnesses and open‐source LLMs run locally (Qwen3 [17], gptoss
[18], Gemma 4 [22]…) on the same task, and map the accuracy–cost frontier (tokens, latency, GPU memory).
2. Which tools really matter? Systematic ablations and attribution of performance to each tool (segmentation, measurements,
2D or 3D VQA such as MedGemma 1.5 [11], foundation models).
3. Is the agent’s reasoning faithful? When the agent justifies its answer, does it describe what actually determined it?
Possible extension, depending on results and interest: the verifiable answers of DeepTumorVQA make it a natural testbed for reinforcement learning of agents [21] (e.g. GRPO on a small LLM).
Main activities
Bibliographic study
Programming in PyTorch
Use of distant GPU servers
Scientific presentations and reporting
Ablation studies and benchmarking
Skills
- Master 2 in machine learning or final year of engineering school.
- Good command of Python and PyTorch; taste for rigorous experimentation.
- A plus: experience with LLMs (Hugging Face, vLLM, tool calling), agent frameworks or evaluation methodology.
- No prior medical knowledge is required. A first experience with 3D imaging (NIfTI, DICOM, MONAI) is appreciated but can be acquired during the internship.
- Fluent French or English; comfortable reading scientific papers in English.
This listing was collected from a public source and is reproduced here for
information only. Always confirm the details on the original posting before applying.
View the original posting