Operating systems (OS) underpin modern IT infrastructure from computers, to smartphones and cloud servers. The OS kernels of these systems are central to their security. Yet their inherent complexity results in a broad attack surface and frequent vulnerabilities, often targeted for denial of service, privilege escalation, or information leakage. While static analysis and fuzzing tools can detect defects in OS kernels, distinguishing exploitable vulnerabilities from benign bugs typically requires manual exploit development, a process that remains labor-intensive. Over the past three decades, attackers have increasingly automated parts of this process, culminating in recent advances in automated exploit generation (AEG) powered by program analysis techniques such as symbolic execution. However, applying these techniques to large complex systems such as OS kernels continues to be challenging. This paper sheds light on the main reasons why it remains challenging to automate exploit generation in OS kernels. We systematize the current knowledge of attacks against kernels in categories, going beyond memory corruption attacks, as well as the relevant threat models and tools used. We categorize existing work along this model to show that gaps exist in many areas. Our analysis helps us identify open problems, in particular the lack of reproducibility across different kernel versions due to the large code base and changing APIs which renders comparisons between different papers difficult. Finally, we propose a set of recommendations for future work in this area.