1. Abstract

     In this paper, we introduce a brand new dataset to promote the study of instance segmentation for objects with irregular shapes. Our key observation is that though irregularly shaped objects widely exist in daily life and industrial scenarios, they received little attention in the instance segmentation field due to the lack of corresponding datasets. To fill this gap, we propose iShape, an irregular shape dataset for instance segmentation. iShape contains six sub-datasets with one real and five synthetics, each represents a scene of a typical irregular shape. Unlike most existing instance segmentation datasets of regular objects, iShape has many characteristics that challenge existing instance segmentation algorithms, such as large overlaps between bounding boxes of instances, extreme aspect ratios, and large numbers of connected components per instance. We benchmark popular instance segmentation methods on iShape and find their performance drop dramatically. Hence, we propose an affinity-based instance segmentation algorithm, called ASIS, as a stronger baseline. ASIS explicitly combines perception and reasoning to solve Arbitrary Shape Instance Segmentation including irregular objects. Experimental results show that ASIS outperforms the state-of-the-art on iShape.

2. Paper

3. Our iShape dataset

In this work, we present iShape, a new dataset designed for irregular Shape instance segmentation. Our dataset consists of six sub-datasets, namely iShape-Antenna, iShape-Branch, iShape-Fence, iShape-Log, iShape-Hanger, and iShape-Wire. As shown in picture below, each sub-dataset represents scenes of a typical irregular shape, for example, strip shape, hollow shape, and mesh shape.

Download iShape dataset (4.5GB):

iShape download URL:

     http://39.105.21.95:9000/ishape/ishape_dataset.tar

Or from our oss URL:

     https://ylshare.oss-cn-shanghai.aliyuncs.com/ishape_dataset.tar

$ md5sum ishape_dataset.tar
# 2b3bd15e6ec762bbc03dddc5e4bc24df

Browse iShape dataset online with visualization: => ishape_dataset

Dataset format: iShape provides both Cityscapes and COCO style instance segmentation annotations.

• Cityscapes style: store as *.png files under directory instance_map. Similar to *_instanceIds.png in Cityscapes dataset, those png file are Height * Width * 16bit. Each pixel value x means that the pixel belongs to the instance ID is x.
• COCO style: It should be pointed out that the COCO style annotations encode masks by Run-Length Encoding(RLE), because polygon can not represent hollow-shaped masks.

Source code about the dataset:

• build_synthetic_ishape: Source code of building iShape synthetic data.
• bpycv: Computer vision utils for open-source CG software Blender.

Dataset license: Public domain (CC0)

4. Our Baseline: Arbitrary Shape Instance Segmentation

We introduce a stronger baseline considering irregular shape in this paper, which explicitly combines perception and reasoning. Our key insight is to simulate how a person identifies an irregular object. Taking the wire shown in Figure 1 for example, one natural way is to start from a local point and gradually expand by following the wire contour and figure out the entire object. The behavior of such ``following the contour'' procedure is a process of continuous iterative reasoning based on local clues, which is similar to the recent affinity-based approaches . Under such observation, we propose a novel affinity-based instance segmentation baseline, called ASIS, which includes principles of generating effective and efficient affinity kernel based on dataset property to solve Arbitrary Shape Instance Segmentation. Experimental results show that the proposed baseline outperforms existing state-of-the-art methods by a large margin on iShape.

Baseline code will release soon.

5. Benchmark

We also benchmark existing instance segmentation algorithms on iShape and find their performance degrades significantly.

5.1 Experimental results

Qualitative results on iShape. We report the mask mmAP of six sub-datasets and the average of mmAP. To be fair, all methods use ResNet-50 as the backbone. ``w/o'' denotes ``without''.

5.2 Qualitative results

Click to open the big image.

Ground Truth	Mask RCNN	SOLOv2	Ours

6. Misc

If you have any questions about iShape, feel free to submit an issue here: issues.