Forget the day to day issues - what happens if IRD investigates in four or five years time. You are required to keep business accounting records for seven years. (In addition trusts are required to keep trust records for the life of the trust - up to 80 years - court cases have shown the importance of this.) Forget other business transactions (although they too are facing face the same issues), think about your bank statements. They are after all the minimum starting point for anyone wanting to check what has happened.
In the old days (I remember it well) we'd go through pages of bank statements - filed and stored for the requisite time. Today many banks encourage their clients to switch to on-line statements, or at least electronic statements. They save everyone (banks and us) so much time and effort - it's generally a win-win situation. Until ...
If they email you documents, you can store those electronically for later access. (You could print them - but that kind of defeats the purpose.) That assumes you don't change email programs without making sure you can still read the old messages. And of course if your email is in the "cloud" (e.g GMail) you have to keep using them and hope they don't have difficulties. Remember the saying "If anything can go wrong, it will. If anything can't go wrong, it will."
Of course most business bank accounts are on-line, usually without statements. I mean, why issue a statement when clients can access the data directly? Sadly some accounts either can't access more than a few months on-line (or the banks charge their usual exorbitant fees to do so). The only safe way to meet your obligations to keep seven years records (prior to the current one) is to take a copy for yourself. And this is probably best done monthly for each account. Exciting - hardly. But you need these.
Of course you then have the problem of being able to read this years later. I won't go through the need for reliable and proven backups again - nor for keeping data in a form that will still be readable years later. But one point is often overlooked. If your original data is in the cloud, then it is good to store the backup locally - on your systems. If your original data is local, then storing it in the cloud provides an off-site backup.
Whatever you choose, backups themselves can fail for many reasons - so use different forms of storing bank statements (and for that matter on-line accounting systems data).