CIPS-Net: Text-Instructed Conditional Nucleus Instance Segmentation Network in Histopathology
Most nucleus segmentation tools return every cell in an image and leave the filtering to the pathologist. CIPS-Net lets a pathologist ask for exactly the cells they need, in plain language, and segments only those.
Nikhileswara Rao Sulake1,†, Sai Manikanta Eswar Machara1,†, Sivaji Retta1, Iyyakutti Iyappan Ganapathi2, Muhammad Owais2, Irfan Hussain2
1 Rajiv Gandhi University of Knowledge Technologies, Nuzvid, India2 Khalifa University Center for Autonomous Robotic Systems (KU-CARS), Khalifa University, UAE† Equal contribution
COMPAYL Workshop, MICCAI Society 2026
A different way to ask for a segmentation
Most nucleus segmentation tools work the same way regardless of what a pathologist actually needs. They segment every nucleus in an image first, sort them into categories afterward, and leave the clinician to manually pick out the cells that matter for the question at hand. That segment everything, then classify approach works reasonably well, but it puts the burden of filtering on the person reading the slide, and it treats every diagnostic question as if it required the exact same output.
CIPS-Net starts from a different premise. Instead of producing one fixed set of masks, it lets a pathologist type an instruction such as segment inflammatory cells or segment dead cells and returns only what was asked for. The goal is not just convenience. Selective segmentation keeps the model's attention on the cell population that is actually relevant to the diagnostic question, which is closer to how a pathologist already works when reading a slide.
Teaching a model to understand instructions
The architecture pairs a DINOv2 vision transformer with Bio_ClinicalBERT, a language model pretrained on clinical text. Image features and instruction embeddings meet at a cross-attention module we call Language-Guided Feature Enhancement, which lets the visual features attend to the specific tokens in the instruction rather than treating the whole sentence as a single flag. Further downstream, a FiLM-based conditioning scheme reshapes the decoder's skip connections at every scale, so the instruction keeps influencing the output through the entire network instead of being applied once and forgotten.
Four lightweight heads then turn those features into instances: one flags nucleus presence, one regresses a distance transform that seeds the instance boundaries, one produces per-pixel embeddings that get clustered into instances, and one classifies each instance's type. It is a fairly involved pipeline, but each head has a narrow job, and that separation is what keeps the model both accurate and fast.
Training without asking for more labels
One practical obstacle to text-conditioned segmentation is that most public pathology datasets, including PanNuke, only ship image-mask pairs, not language descriptions. Rather than collect a new dataset, we generated instructions automatically from the existing class labels, and, more importantly, trained on every possible non-empty subset of classes present in each image. An image with three annotated cell types yields seven different training examples, one for each combination a pathologist might ask for. This permutation-based strategy is what actually teaches the model to respect the instruction instead of segmenting everything and ignoring the text.
What the numbers say
On PanNuke, evaluated with the standard three-fold cross-validation protocol, CIPS-Net reaches a multi-class panoptic quality of 0.4846 and a binary panoptic quality of 0.6217. That is competitive with strong unconditional baselines such as CPP-Net, and it clearly outperforms the vision-language segmentation baselines we compared against, including LViT and Grounding DINO adapted to this task. What we found more interesting is where the gains showed up: CIPS-Net's performance on the Dead and Inflammatory classes, both rare and diagnostically important, improved noticeably over the strongest unconditional baseline, a reasonable sign that focusing a model's attention on a specific instruction helps most where it is needed most.
Efficiency held up too. CIPS-Net needs about 11 percent fewer computations and runs roughly 20 percent faster than the best-performing unconditional baseline we tested against, while its trainable parameter count stays modest since the text encoder remains frozen during training.
Why we think this matters
None of this replaces a pathologist's judgment, and it was never meant to. The intention is narrower: reduce the amount of manual filtering a busy diagnostic workflow requires, and make the segmentation tool responsive to the specific question being asked rather than a fixed, one-size-fits-all output. CIPS-Net will be presented at the COMPAYL Workshop, MICCAI Society 2026, and the code will be released publicly alongside the paper.