OK, actually I just got mine working great as far as I can tell. Yesterday no matter what I tried I kept getting errors. Once I gave up and just cleaned out my FA folder directory and copied BACK over the previous folders and files I had manually copied as a back up yesterday, I was able to restore things to a "pre-update" state in version 2.3.19 .
So today instead of uploading the archive update file (2.3.21) and extracting in my current folder, I made a new folder, uploaded into it and extracted in the same which automatically gives me a back up copy of the new files. I looked at permissions, made sure my permissions matched the (same name) working files & folders I had, then I copied the entire list of folders and files for version 21 over to my current live working directory for FA. Once that took place, only the relevant files were updated and databases remained the same as the security fix did not include database files anyway. I refreshed my directory on my server and then browsed to my web url to see what would happen. My login page came up, I logged in, and no problems so far. Version shows updated to 2.3.21 and I can click variuos menus, view reports, orders, invoices, items, bank accounts, etc. with no errors.
I am not sure what I did different, but after several times yesterday trying to extract the archive and no luck - plus permissions and files were not showing as updated consistent, I tried the method above and it worked.
As for permissions for now, my main folders are set to 755 on the server FA directory. My files (.htaccess, config.default.php, etc.) are set to 644 with exception of my config.php and config_db.php are both set to 440.