Finally I bit the bullet and made an rsync script in bash to upload all the contents of my site to the server. To understand why this is a big deal to me you need to know some context of how I've managed this site in the past.
Back when I would make iterations of this site I would create them locally, tar them up, upload them to some file host, go on to the server, wget them down then remove the old site and untar the archive and boom a new site was uploaded. Now, this method worked but was manual and time consuming. I found an old archive of a really old version of the site one time and accidentally untarred it in the same directory as my site which nuked all the files and that was a big motivator for me to finally make this script and I really don't regret making it.
Now I can make a change to a local copy then just run the script, do some auth things then it uploads it to the server. This is perfect