Yii 2 404 Not Found

After setting up a new application using Yii 2 Advanced Project Template, the very first time you want to load your new controller, if you are met with a 404 Not Found, one very likely cause is that you do not have a .htaccess in your web folder.