We present a policy based reinforcement learning scheduling algorithm with high level deadlock detection for job-shop discrete manufacturing systems without buffer being equipped. Deadlock is a highly undesirable phenomenon resulting from resource sharing and competition. Hence, we first propose detection algorithms for second and third level deadlocks. Subsequently, based on these high level deadlock detection algorithms, a new policy based reinforcement learning scheduling algorithm is developed in the context of buffer-less job-shop systems. Applying our reinforcement learning approach into scheduling algorithm to a set of 40 widely-used buffer-less job shop benchmark, satisfactory makespan can be obtained, which, to our knowledge, have never been published before. It is safe to conclude that our policy based reinforcement learning scheduling algorithm can be applied to other discrete event systems (e.g., computer operation systems, communication systems, and traffic systems).