So, the server has been running well for the last 12+ hours. The highest spike in the last hour was to 8 and nothing came close to 8 since 7pm last night (also around 8) and prior to that, it was while we were still making changes.
So, nobody should now get any server too busy messages.
This said, we're going to downgrade the server back to it's original spec (host gave us temporary extra cores - 6 in stead of the 4 we're paying for), and we'll monitor then.
We will also be upgrading to the latest version of mysql, which also has performance improvements. So, perhaps things will be fine now. I am still monitoring. Report any issues in this thread.