Automatic Detection of Android Steganography Apps via Symbolic Execution and Tree Matching

Abstract

The recent focus of cyber security on automated detection of malware for Android apps has omitted the study of some apps used for "legitimate" purposes, such as steganography apps. Mobile steganography apps can be used for delivering harmful messages, and while current research on steganalysis targets the detection of stego images using academic algorithms and well-built benchmarking image data sets, the community has overlooked uncovering a mobile app itself for its ability to perform steganographic embedding. Developing automatic tools for identifying the code in a suspect app as a stego app can be very challenging: steganography algorithms can be represented in a variety of ways, and there exists many image editing algorithms which appear similar to steganography algorithms. This paper proposes the first automated approach to detect Android steganography apps. We use symbolic execution to summarize an app’s image operation behavior into expression trees, and match the extracted expression trees with reference trees that represents the expected behavior of a steganography embedding process. We use a structural feature based similarity measure to calculate the similarity between expression trees. Our experiments show that, the propose approach can detect real world Android stego apps that implement common spatial domain and frequency domain embedding algorithms with a high degree of accuracy. Furthermore, our procedure describes a general framework that has the potential to be applied to other similar questions when studying program behaviors.