Ok, Understand, this is all strictly IMHO.....That being said, I have been a gun nut for over 35 years and have owned literally hundreds of firearms over the course of my life.
For starters, Inexpensive is a relative term, but I will try and stick to firearms that are under $150 in price. I want to make it clear here that I am not including hand guns in this piece, That's an entire other kettle of fish. So, You need to ask yourself a question first...(And it's NOT do I feel lucky, Punk)..... The big question is: "What am I going to use this firearm for?
Pretty much, best bang for your buck is a 12 Gauge shot gun. 12 Ga are relatively inexpensive, common, and come in so many possible loads that you literally can own a 12 Ga and not own ANY other arm. 12 Ga slugs can knock over any game on earth if properly placed on target, OO Buck is a great medium sized game and self defence load if you're out doors, Number 6 is good for birds, and also a good choice for in the house home defence. (lethal to your target, but tends to be low penetration, so it doesn't punch through walls and accidentally take out innocent people).
There are also many other sized loads for various types of hunting. I would stick with 2 3/4ths " shells for most work, as they are cheaper, lower recoiling, and while you can't stuff 3" or 3 1/2 inch Magnums into shotguns made for 2 3/4ths shells, you sure can put 2 3/4ths shells into magnum guns!!!!
Most of that above applies to the 20 gauge shot gun as well. I'm a HUGE fan of the 20 gauge. I happen to think that it does pretty much everything as well as a 12, for about 2/3rds of the recoil. Feel free to disagree. All I'll say is that I've never shot anything with a 20 gauge that got back up.
On the guns themselves. Small gun stores and pawn shops almost always have a rack of used guns. This is where you want to go. While you may be able to pick up a used pump action gun at a reasonable price, I highly recommend instead grabbing a single shot top break shot gun. Over the last 100+ years, there have been literally hundreds of models of this type made. The most commonly found will be Harrington and Richards, NEF, J. Stevens, Ithica, and Savage. Most of these guns while inexpensive, were well made, and most importantly RELIABLE!!! (think about it, VERY FEW moving parts means far fewer things to break). And while some folks will argue you into getting a Pump gun for it's higher rate of fire, realistically, how often are you going to be in a situation where volume of fire is an issue unless you're a cop or soldier? best thing about a used single shot. It is very often possible to find them for as little as $50. The cheapest USED pump is going to run you almost $100 more. and let me tell you, $100 buys A LOT of shot gun shells!!!
While we are at the used gun rack. The next best choice for many reasons is a good .22 I recommend either a bolt action. or single shot here, but in this case, there are a goodly number of perfectly adequate semi-autos out there. This is the one category where it is possibly, if your watching sales closely to get a BRAND NEW gun at one of the big box chains for under $150. I have recently seen a Marlin model 60 for sale at K-Mart for $149.99. The .22 Long Rifle cartridge is almost the PERFECT pot hunting load for anything about medium dog sized or smaller. It's probably put more rabbits and squirrels into cook pots than every other caliber combined. .22 LR is also the single cheapest ammo out there, with a 100 round box of good quality high velocity loads going for about $8 and change. It has to all intents and purposes NO RECOIL, It's also a stone HOOT to shoot. And nothing can hone your rifle skills quite like a casual day of knocking over soda cans with a .22. Don't however let it's diminutive size fool you. a .22 will carry for over a mile, and can kill someone just as dead as a much larger round, so ALWAYS know what's behind what your shooting at...(A good rule of thumb with ANY firearm)
You will note that I prefer a single shot or bolt action here again. The reason for this is that .22 ammo is very dirty. Even the number one most popular .22 in America the Ubiquitous Ruger 10/22 will jam eventually from fouling. Singe shot and bolt actions, being purely mechanical, have a much lower probability of reliability problems. Once again, KISS rules..... ("Keep It Simple Stupid" for those unfamiliar with the acronym) The other issue here is that the vast majority of the Semi-auto .22 your going to find on the rack have probably been fired THOUSENDS of times, and consequently are more worn out. It takes WORK to put THAT many rounds through a bolt action or single shot.
Lastly, also off the use rack. Badly sporterized old military rifles. Especially Mausers, Lee-Enfields, and Moisin-Nagants. Often, these guns look like hell, but far more importantly, they're still mechanically reliable. With the Mausers, try and stick to the ones in the more common calibers. 8mm Mauser, 7x57 Mauser, or 6.5x55 Swedish. Trying to find ammo for some of the other calibers can be a stone bitch. Also, every one of those rounds is just fine for Deer, or honestly pretty much any other game in North America. The Lee-Enfield has a marvelous action, holds ten rounds to most other bolt actions 5, and has a reputation for reliability and accuracy. Plus, .303 British is fairly easy to find most places. The Moisin-Nagant like most Russian designs is built solid as a bank vault. I have NEVER seen one jam, or fail to fire... NEVER. it's round, the 7.62x54R is a good one, quite accurate, and still fairly cheep and plentiful.
There are other possible choices, but for under $150, these are your best bet for reliability, low cost, inexpensive and easy ammo availability, no brainer functionality, and high lethality.
Fee free to ping me with questions..