Yixing value comes down to basically two things (though these are linked).
1) Craftsmanship
2) Clay quality

Craftsmanship
Well-made pots will cost more. The better the craftsmanship, the higher the price on the pot will be.
Part of the cost of craftsmanship will definitely be the renown of the potter, which is not really all that important. However, it does have an effect on the price, so it must be mentioned. The most famous potters should make the best quality pots, so these command higher prices. However, you’re right- a first time pot is not really concerned with this.
There are also some very practical reasons to look for a well-made pot. Well-made pots make the best tasting tea for a couple of reasons. Pour (spout construction and execution)
A pot’s spout effects the rate at which the tea is poured out of the pot. A poorly-made spout will often poor out too slowly. These will also dribble, which is basically just a messy annoyance. You just sunk some change into a tool you’re going to be using for at least a decade- why settle for something that dribbles?

Caring for your pot
If you’re getting an Yixing pot and you want it to give you back the best flavor possible, you’re going to have to care for it properly. The means saving a little bit of each steeping to “feed” the pot (pour over the pot to help more tea get into the pores), letting the pot dry completely between uses (to avoid mold and to help the flavor “set” into the pot… if you use a wet post, then alot of the flavor will just get washed away instead of building up nicely), etc etc etc.
The finer your pot, the more likely you are to use it, and the more likely you are to actually invest the time in caring for the pot properly.

WATCH OUR VIDEOS ON HOW TO CARE FOR YOUR TEAPOT AS A PROFESSIONAL COLLECTOR

Tips, suggestions and how-to

If you’re not interested really in yang-ing a pot and devoting the time to raise one that will give back the best flavor, you might be better off sticking with a smaller glazed pot. This will still give you the power of short steepings and better heat-holding-abilities, but it will give greater flexibility in terms of what kind of tea you can make in it. Since it’s glazed, you can put anything in there! These are also easier to find, plus you cut out the added cost of clay quality, clay rarity, and the fame of the potter.

The Lid
High-quality pots should have a lid that fits well into the body of the pot. This makes for a better heat seal. One of the benefits of making tea in an yixing pot is that they hold heat extremely well and keep your tea steeping at a constant temperature. If the lid fits poorly, the pot will loose heat quickly. It is also at a great risk of falling out of the pot and chipping (never fun- you don’t want your tool to break), or of being a generally dribbley pot. Also, the lid construction and fit is a great indication of quality over-all.Clay Quality
This also greatly influences the price of a pot. Three contributing factors here are clay quality, rarity, and pot size.

Size
The bigger the pot is, the more clay it used. That’s pretty simple. I always recommend small pots for yixing. For example, most of my pots are around 6-8oz, and they can serve 6 comfortably. Anything bigger, and you might as well be using a western-style pot.
As a tool, yixings are designed for gong-fu ceremony, and it is through gong-fu that they give back the best flavor. I find filling the pots with more leaves than usual and doing quick steepings is where yixing gives you back the most interesting things. You get to experience snapshots in time of the full range of a tea’s flavor, instead of compressing then and simplifying them into just one taste. A smaller pot is ultimately more cost effective, since you can get the same experience (20+ steepings, full flavor, etc) using less leaves.

Clay rarity

This just reflects the simple demands of the market. The more rare a clay is, the more expensive it is to get your hands on it. The true rare clays usually fall into the hands of the best craftsmen because 1) they can pay for it and 2) the people who have the clay would rather it get used for a nice pot than a crappy pot. This general trend also pushes up the prices for rare clays.

There’s no need to go for a real zhu ni or huang ni pot on your first pot, unless you’re an extreme-collector-type. Clay Quality

A clay is good quality if it will grow well over time and (eventually.. the ultimate goal) if it will give back goodness to the teas you steep in the pot. Higher quality clays are greedy- they are more porous. As a result, they will suck up a lot more tea. Since they suck up more tea, they will also be giving back more in terms of flavor and texture than a pot that is not greedy.

The higher quality clays will of course be just a little more because of demand. Interestingly, greedy pots will also become the most beautiful over time if you take care of them well.

Also, this kind of clay holds heat much better, and so will brew tea better.

Another unrelated thing:
Aesthetics (how nice does the pot look?)
This may seem like a silly thing to pay for on the surface, but it is actually very closely tied to making the most delicious tea.
The more beautiful a pot is to you, the more often you’re going to want to use it. These kinds of things also make using the pot more enjoyable, which (again) means you’ll be more interested in using this tool. A tea pot is, as you say, an extremely fine and powerful tool for making tea, but it is only as good as the time you put into it. The more you use it, the better it becomes.
Also, the more you enjoy using the pot, the better the tea will taste. I’m serious: the placebo effect and the ramifications of ceremony cannot be ignored here if you’re seriously considering investing in an yixing pot. Here’s why.
Yixing pots give the best flavor when used in gong-fu. Even really casual gong-fu enforces just a little bit of ceremony on the drinkers, because you drink short, small steepings out of small cups. Small cups means you have to sip your tea (savor it, taste it), instead of just drinking out of a mug. It’s the difference between drinking to qunch thirst and drinking to appreciate tastes, flavors, textures. This is tea as an experience as opposed to tea as a beverage. Tea that is tasted with care will always taste better, because you are paying attention to it. You are looking for the good things in the cup, and so they are easier to find. It just tastes better, which is awesome.
If your yixing pot can in anyway contribute to this tasting, this ultimately aesthetic experience of the senses, it will in the end also make the tea taste better on a psychological level. Something coming out of a lovely vessel will make your more positively disposed towards the tea that’s coming out of it. This placebo effect should in turn help you taste more interesting things in the tea by helping to make you just a little more open minded and positive.

As you say- an yixing pot is at heart an extremely fine, specialized tool. Compare it to a fine chef’s knife. If you’re going to invest in a good knife and you really care about having one that cuts well for you, then you might as well get a good one. You also have to make sure you’re caring for it properly. For a knife, this will mean using a wetstone and a steel often to keep the blade sharp and straight, not just an automate sharpener once or twice a year. This is what I meant when I spoke of raising and growing (yang-ing) a tea pot. I was speaking of the technical act of actually caring for the teapot well in order to make good tea, not of fetishizing or anthropomorphizing an object unnecessarily.