FileZilla FTP Client - http://filezilla-project.org/
A FTP client to upload documents from your computer to your web server. Also supports SFTP for secure transfering of files.
xampp - http://www.apachefriends.org/
A bundled installer of Apache, PHP, MySQL and a few other programs. This allows you to set up a server on your computer accessible through http://localhost which saves you having to upload the files to a server during development.
Notepad++ - http://notepad-plus.sourceforge.net/
GIMP (GNU Image Maninpulation Program) - http://www.gimp.org/
A worthy alternative to the hugely expensive Photoshop.
Paint.net - http://www.paint.net/
Based on the simplicity of MS Paint, Paint.net takes this a lot further with the introduction of many tools that are present in GIMP and Photoshop.