Exploring the evolution of active galactic nuclei with the next generation of astronomical surveys