As Clayton points out, you’re on the right track but you’re a bit off. It is true that money the government borrows is money that the private sector cannot. This is usually what is being referred to with the term “crowding out”. But this does not mean money that is borrowed will never be returned.
But along these lines, if you really want to get down to it, yes, you can break down government financing into smaller and smaller categories. On the surface there are three ways government finances its operations: Taxing, borrowing, and printing (money). You can fuse printing into taxing, because as Milton Friedman and virtually every other economist worth his salt has pointed out, money printing (inflation) is simply an invisible tax…invisible because it is not a tax on nominal dollars, but on purchasing power…which is really what matters. So then you’re left with taxing and borrowing. But even then you can fuse borrowing, as interest payments must be made, and (in theory), the principle must be repayed…and of course, where does government money come from: either taxes or borrowing. So you either end up with a Ponzi scheme (which will eventually collapse), or more taxes. Most of the time, both.
So ultimately, yes, all government spending is financed by a tax on the private sector, which means that yes, in a sense, all government spending is a tax.