A penny saved is a penny earned. (Bejamin Franklin)
You can't run a business without accounting - at least not without headaches. You can run a business without becoming an accountant - but you do need to understand the basics. You might think your software can do it all - but it is only as good as the way it's used. You can get someone else to do it all. That will cost you, but free you up for more important tasks. However it may still get you offside with IRD (can be expensive) if the other person gets it wrong.
Even if you did that, you'd still have to collect and collate all of the information and deliver it to the person doing the accounts. It's always a good starting point to keep separate bank accounts for the business from your personal accounts. (These days most accounting software offers bank feeds to save time.) Plus you're still responsible for accounts and taxes, even when you use a third party. (That's why it can help if they have professional indemnity insurance.)
And what does accounting cover anyway. Well the first thing most people are interested in is selling stuff and getting paid for it - sales and debtors. The other side of this is buying stuff and paying for it - purchases and creditors. And then there's keeping track of where money's coming from and where it's going to. Generally that's done by some sort of cash book.
Of course some of the things raise other issues. For example, not everything that's purchased is for sale. Some of it is to keep the business running (e.g. power), and some of it is used to run the business for several years (e.g. assets such as a car). These things often involve finance arrangements, and the main cost is not allowed directly as a business expense, but through depreciation allocated over the estimated useful life of the asset.
Then there are other complications required by the IRD. GST is one such thing, as is personal "wages" from your business. And then there's special rules around items with both business and private benefits, such as FBT.
All in all it's complex - despite the government's wonderful "tax simplification". It's certainly simpler than say the USA - even our GST is simpler than across the ditch. But lack of knowledge is no defence - so relying on advice from "down the pub" is dangerous.
When you want help, but how can you get someone who knows their stuff and will look after your interests first rather than theirs? Many people look for an accountant - which is a good start. If you think this is complex, many people in the USA use a tax attorney, reflecting how complex their systems are.
But when you say accountant, what do you actually mean? It's not illegal for anyone to call themselves an accountant (some bookkeepers call themselves accountants), and some accountant's who aren't part of either professional body do a good job. But how can you tell if someone is a good accountant or best avoided. I've seen accounts done by people who don't even bother to check they balance (under double-entry accounting one side says where funds are from, and the other where they have gone), let alone make sense and have reasonable checks on the transactions.
Some of us are either Certified Public Accountants or Chartered Accountants - but so what? Well, to be able to call ourselves that we have certain standards to maintain. We have to be members of CPA Australia - the dominant body in Australia but relatively new to NZ, or of CAANZ (TransTasman body that replaced the Australian and NZ Institutes of Chartered Accountants). They have rules and standards to follow, and disciplinary committees to enforce them. They require their members to have professional indemnity insurance. That doesn't mean we are perfect (the occasional court or disciplinary case is testimony to that), or that we all offer the same services. But it does ensure you have a higher chance of getting a minimum standard of service.