Improving function signature in malware analysis using neural networks

Binary code similarity detection is a critical task in numerous security applications such as malware analysis, bug search, and software theft detection. This thesis explores the applications of the SAFE (Self-Attentive Function Embeddings) tool to enhance the SEMA-Toolchain by implementing a novel method for detecting similar functions in binary code. SAFE leverages a self-attentive neural network to generate function signatures, which are then used to identify similar functions across different binaries. The proposed method is evaluated in samples of several malware families such as Warzone and Satan. We demonstrate that our approach is capable of detecting similar functions in binaries more effectively and accurately than the current method used in the SEMA-Toolchain. Furthermore, we show that our method can detect common functions in different malware families. Finally, we demonstrate that our method can be used to improve the SEMA-Toolchain and is able to detect similar functions in binaries compiled from the same source code but with different compilers and compiler optimizations. Finally, we discuss the limitations of our work and propose future research directions to further enhance the performance of binary code similarity detection.