Abstract: Artificial neural networks (ANNs) have made great strides in the field of remote sensing image object detection. However, low detection efficiency and high power consumption have always been significant bottlenecks in remote sensing. Spiking neural networks (SNNs) process information in the form of sparse spikes, creating the advantage of high energy efficiency for computer vision tasks. However, most studies have focused on simple classification tasks, and only a few researchers have applied SNNs to object detection in natural images. In this study, we consider the parsimonious nature of biological brains and propose a fast ANN-to-SNN conversion method for remote sensing image detection. We establish a fast sparse model for pulse sequence perception based on group sparse features and conduct transform-domain sparse resampling of the original images to enable fast perception of image features and encoded pulse sequences. In addition, to meet accuracy requirements in relevant remote sensing scenarios, we theoretically analyze the transformation error and propose channel self-decaying weighted normalization (CSWN) to eliminate neuron overactivation. We propose S3Det, a remote sensing image object detection model. Our experiments, based on a large publicly available remote sensing dataset, show that S3Det achieves an accuracy performance similar to that of the ANN. Meanwhile, our transformed network is only 24.32% as sparse as the benchmark and consumes only 1.46 W, which is 1/122 of the original algorithm’s power consumption.

S3Det：一种基于人工-脉冲神经网络转换的遥感影像目标快速检测模型

[1]Azimi SM, Vig E, Bahmanyar R, et al., 2018. Towards multi-class object detection in unconstrained remote sensing imagery. Proc 14^th Asian Conf on Computer Vision, p.‍150-165.

[2]Chen GH, Pei GS, Tang Y, et al., 2022. A novel multi-sample data augmentation method for oriented object detection in remote sensing images. Proc IEEE 24^th Int Workshop on Multimedia Signal Processing, p.1-7.

[3]Chen K, Wang JQ, Pang JM, et al., 2019. MMDetection: open MMlab detection toolbox and benchmark. https://arxiv.org/abs/1906.07155

[4]Chen L, Zhang F, Guo W, et al., 2023. SFTN: fast object detection for aerial images. IET Image Process, 17(13):3897-3907.

[5]Cheng G, Zhou PC, Han JW, et al., 2016a. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Trans Geosci Remote Sens, 54(12):7405-7415.

[6]Cheng G, Zhou PC, Han JW, 2016b. RIFD-CNN: rotation-invariant and Fisher discriminative convolutional neural networks for object detection. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2884-2893.

[7]Ding J, Xue N, Long Y, et al., 2019. Learning RoI Transformer for oriented object detection in aerial images. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.2844-2853.

[8]Eldar YC, Kutyniok G, 2012. Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge, UK.

[9]Everingham M, van Gool L, Williams CK, et al., 2010. The PASCAL Visual Object Classes (VOC) challenge. Int J Comput Vis, 88(2):303-338.

[10]Gong MG, Li JZ, Zhang YR, et al., 2022. Two-path aggregation attention network with quad-patch data augmentation for few-shot scene classification. IEEE Trans Geosci Remote Sens, 60:4511616.

[11]Han JM, Ding J, Li J, et al., 2022. Align deep features for oriented object detection. IEEE Trans Geosci Remote Sens, 60:5602511.

[12]He X, Ma SP, He LY, et al., 2022. High-resolution polar network for object detection in remote sensing images. IEEE Geosci Remote Sens Lett, 19:6000605.

[13]Horowitz M, 2014. 1.1 Computing’s energy problem (and what we can do about it). Proc IEEE Int Solid-State Circuits Conf Digest of Technical Papers, p.10-14.

[14]Hu YF, Zheng Q, Jiang XD, et al., 2023. Fast-SNN: fast spiking neural network by converting quantized ANN. IEEE Trans Patt Anal Mach Intell, 45(12):14546-14562.

[15]Huang ZC, Li W, Xia XG, et al., 2022. A general Gaussian heatmap label assignment for arbitrary-oriented object detection. IEEE Trans Image Process, 31:1895-1910.

[16]Jiang YQ, Tan ZY, Wang JY, et al., 2022. GiraffeDet: a heavy-neck paradigm for object detection. https://arxiv.org/abs/2202.04256

[17]Jiang YY, Zhu XY, Wang XB, et al., 2018. R²CNN: rotational region CNN for arbitrarily-oriented scene text detection. Proc 24^th Int Conf on Pattern Recognition, p.3610-3615.

[18]Kim S, Park S, Na B, et al., 2020. Spiking-YOLO: spiking neural network for energy-efficient object detection. Proc 34^th AAAI Conf on Artificial Intelligence, p.11270-11277.

[19]Komárek A, Lesaffre E, 2008. Generalized linear mixed model with a penalized Gaussian mixture as a random effects distribution. Comput Stat Data Anal, 52(7):3441-3458.

[20]LeCun Y, Bottou L, Bengio Y, et al., 1998. Gradient-based learning applied to document recognition. Proc IEEE, 86(11):2278-2324.

[21]Li Y, He X, Dong YT, et al., 2022. Spike calibration: fast and accurate conversion of spiking neural network for object detection and segmentation. https://arxiv.org/abs/2207.02702

[22]Lin TY, Maire M, Belongie S, et al., 2014. Microsoft COCO: common objects in context. Proc 13^th European Conf on Computer Vision, p.740-755.

[23]Lin TY, Goyal P, Girshick R, et al., 2017. Focal loss for dense object detection. Proc IEEE Int Conf on Computer Vision, p.2999-3007.

[24]Liu WX, Luo B, Liu J, et al., 2022. Synthetic data augmentation using multiscale attention CycleGAN for aircraft detection in remote sensing images. IEEE Geosci Remote Sens Lett, 19:4009205.

[25]Liu ZK, Wang HZ, Weng LB, et al., 2016. Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds. IEEE Geosci Remote Sens Lett, 13(8):1074-1078.

[26]Ma JQ, Shao WY, Ye H, et al., 2018. Arbitrary-oriented scene text detection via rotation proposals. IEEE Trans Multim, 20(11):3111-3122.

[27]Maass W, 1997. Networks of spiking neurons: the third generation of neural network models. Neur Netw, 10(9):‍1659-1671.

[28]Merolla PA, Arthur JV, Alvarez-Icaza R, et al., 2014. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197):‍668-673.

[29]Ming Q, Zhou ZQ, Miao LJ, et al., 2021. Dynamic anchor learning for arbitrary-oriented object detection. Proc 35^th AAAI Conf on Artificial Intelligence, Electronic Network, p.2355-2363.

[30]Rathi N, Roy K, 2023. DIET-SNN: a low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE Trans Neur Netw Learn Syst, 34(6):3174-3182.

[31]Rueckauer B, Liu SC, 2021. Temporal pattern coding in deep spiking neural networks. Proc Int Joint Conf on Neural Networks, p.1-8.

[32]Rueckauer B, Lungu IA, Hu YH, et al., 2017. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front Neurosci, 11:682.

[33]Sinha D, El-Sharkawy M, 2019. Thin MobileNet: an enhanced MobileNet architecture. Proc IEEE 10^th Annual Ubiquitous Computing, Electronics & Mobile Communication Conf, p.280-285.

[34]Vaswani A, Shazeer N, Parmar N, et al., 2017. Attention is all you need. Proc 31^st Int Conf on Neural Information Processing Systems, p.6000-6010.

[35]Wang A, Chen H, Liu LH, et al., 2024. YOLOv10: real-time end-to-end object detection. https://arxiv.org/abs/2405.14458

[36]Xia GS, Bai X, Ding J, et al., 2018. DOTA: a large-scale dataset for object detection in aerial images. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3974-3983.

[37]Xie XX, Lang CB, Miao SC, et al., 2023. Mutual-assistance learning for object detection. IEEE Trans Patt Anal Mach Intell, 45(12):15171-15184.

[38]Xie XX, Cheng G, Li QY, et al., 2024a. Fewer is more: efficient object detection in large aerial images. Sci China Inform Sci, 67(1):112106.

[39]Xie XX, Cheng G, Rao CF, et al., 2024b. Oriented object detection via contextual dependence mining and penalty-incentive allocation. IEEE Trans Geosci Remote Sens, 62:5618010.

[40]Xie XX, Cheng G, Wang JB, et al., 2024c. Oriented R-CNN and beyond. Int J Comput Vis, 132(7):2420-2442.

[41]Xiong YY, Liu HX, Gupta S, et al., 2021. MobileDets: searching for object detection architectures for mobile accelerators. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3824-3833.

[42]Yang X, Yan JC, 2022. On the arbitrary-oriented object detection: classification based approaches revisited. Int J Comput Vis, 130(5):1340-1365.

[43]Yang X, Yang JR, Yan JC, et al., 2019. SCRDet: towards more robust detection for small, cluttered and rotated objects. Proc IEEE/CVF Int Conf on Computer Vision, p.8231-8240.

[44]Yang X, Yan JC, Feng ZM, et al., 2021a. R3Det: refined single-stage detector with feature refinement for rotating object. Proc 35^th AAAI Conf on Artificial Intelligence, p.3163-3171.

[45]Yang X, Yan JC, Ming Q, et al., 2021b. Rethinking rotated object detection with Gaussian Wasserstein distance loss. Proc 38^th Int Conf on Machine Learning, p.11830-11841.

[46]Yao M, Zhao GS, Zhang HY, et al., 2023. Attention spiking neural networks. IEEE Trans Patt Anal Mach Intell, 45(8):9393-9410.

[47]Zhang C, Lam KM, Wang Q, 2023. CoF-Net: a progressive coarse-to-fine framework for object detection in remote-sensing imagery. IEEE Trans Geosci Remote Sens, 61:5600617.

[48]Zhang GJ, Lu SJ, Zhang W, 2019. CAD-Net: a context-aware detection network for objects in remote sensing imagery. IEEE Trans Geosci Remote Sens, 57(12):10015-10024.