The popular discussion of economics divides into macro and micro, with those familiar with the former tending to adopt a slightly patronising attitude towards the latter (the legacy of Keynes' de haut en bas style), which in turn sees them map onto a left-right spectrum. This is misleading, not only because macroeconomics has historically been an attempt to reconcile classical liberalism with the reality of the state as an economic actor, thereby excluding the need for a socialist or Marxist analysis, but because it tends to omit large swathes of the real economy. A famous example was the lack of attention paid to finance and banking as a systemic vulnerability prior to 2008. But an even more prevalent omission in the popular macroeconomic discourse, which was once central to economics in the days of Adam Smith and David Ricardo, is the role of rent.
Smith laid out the basic proposition: "The rent of land, therefore, considered as the price paid for the use of the land, is naturally a monopoly price. It is not at all proportioned to what the landlord may have laid out upon the improvement of the land, or to what he can afford to take; but to what the farmer can afford to give." Ricardo formulated this as a general law, to wit that the rent of a piece of land will equal the additional monetary gain of its productive use relative to the production of a rent-free piece of land. In other words, tenant farmers will desert high-rent land if the rent exceeds the marginal loss they would incur by farming a rent-free piece of land.
This theory was useful in an era when many people were tenant farmers and when colonialism was bringing marginal (i.e. rent-free) land into production, so the idea of farmers upping sticks to find a more economically advantageous plot wasn't as unrealistic as it seems to us today. Obviously the externalities of colonialism were ignored while rent was seen as a product of natural endowment - the gift of heaven - and the industry of white colonisers (cf Locke). Subsequent attempts, e.g. by Marx, were made to focus on the capital investment of land, its improvement in Smith's terms, and how natural endowment in reality gives rise to rentierism, i.e. monopoly exploitation, notably in the area of patents and technical innovation (as theorised by Joseph Schumpeter).
The one area of rent that has tended to receive far less attention from economists, in terms of explaining what determines its price, is the rent of property, and specifically houses and flats. This might seem odd given how large rent looms in our lives. Even if you have bought a property or are currently paying a mortgage to do so, you are subject to rent insofar as house prices will always reflect the equivalent contract rent - i.e. what you could get if you let it over the same period as a typical mortgage (hence buy-to-let). Many people assume that the dynamic of this relationship works in the opposite direction: high house prices lead to high rents, and that rising house prices are simply the consequence of demand outstripping supply, hence the arguments that we should ease planning restrictions or curtail immigration, but this ignores that there is no shortage of empty or under-occupied houses and flats across the country. So what determines rent?
The law of supply says that more goods will be produced at higher prices. In other words, if demand for a commodity grows, thereby pushing up the price, producers will increase output to take advantage of the larger demand and thus fatter profit margins. The law of demand says that at higher prices demand falls. So once supply of that commodity exceeds demand, following that increase in output, prices will fall back to their notional equilibrium level. This simplistic model obviously ignores a lot of real world frictions and contraints. For example, not all commodities can be rapidly produced at a higher rate, e.g. by adding shifts or converting existing production lines. Likewise, if the market is cartelised there may be a reluctance among producers to increase output excessively. OPEC is the obvious example here.
In the case of housing, there are real constraints such as restrictive planning regulations and limited real resources (builders and building materials), but the biggest determinant is the reluctance of volume builders to over-supply the market and so depress prices. In this context, the state is a volume builder that has taken a self-denying ordinance to maintain house prices, both for owner-occupiers and landlords, which is why the UK government is so reluctant to build council houses despite the pressing need, and why US liberals like Ezra Klein and Derek Thompson argue that "abundance" can be achieved by simply rolling back regulations and striking out building codes, which provides an easy excuse to ignore capitalist realities in favour of a technocratic can-doism.
The "law" of demand is also undermined by necessity. In other words, there are certain things we have to buy, at least at a minimal level, such as shelter, food and clothing, lest we risk injury or death (self-sufficiency is not a practical strategy for most people and a return to a subsistence economy would result in mass starvation). We cannot realistically choose not to buy shelter, preferring to spend our money on first editions or champagne, so demand cannot fall to such low levels that prices must drop. Equally, we cannot easily cut back on the amount we spend on shelter, unlike certain other necessities such as food or clothes. We can skip meals or wear socks with holes in them, but we can't decide to move to a cheaper flat for a month and then back again to ease our cashflow.
When we talk of "the housing market" (singular) we are dealing in a fantasy. In reality, there are hundreds, if not thousands, of geographically limited housing markets, which estate and letting agents understand only too well. Goods (i.e. houses and flats) cannot be moved from one market to another, so prices must always reflect local circumstances. We also cannot easily choose to buy from alternative suppliers in cheaper markets. If I work in London but can't afford the rent, there's no point renting a flat in Sunderland. And if I got an equivalent job in Sunderland, it might not pay well enough to allow me to rent there either.
When house prices or rents do fall, that is typically because of a relative over-supply in a limited geographical market. But when this happens it is rarely because the quantum of supply rapidly increases. Instead it is because the quantum of demand rapidly falls. The obvious examples are all around us: areas that saw deindustrialisation in the 1980s with the result that the population shrank. But the fall in house prices and rents in those areas also reflects the lower average income of the remaining population: deindustrialisation typically took away above-average wage jobs, and they were above-average in most cases due to the strength of trade unions in heavy industries like coal, steel and shipbuilding. Outside these geographic exceptions, house prices and rents rarely if ever fall, something that cannot be explained away as price "stickiness" or the lower bound of a zero return on capital.
Rents then will always reflect "what the market can bear", which is a polite way of saying that landlords will push prices up to their maximum: the point where tenants can just about afford them, assuming they're willing to limit expenditure on other goods, which may be discretionary, such as entertainment, but may also be necessities, such as food and clothing. The "cost of living crisis" due to the recent spike in food and energy costs shouldn't distract from the fact that the prices of these other necessities are, in real terms, a fraction of what they were 50 years ago. That rents have grown over this period is not because people have felt that housing was a better choice for their discretionary expenditure, despite the relentless media propaganda, but because landlords have, in Smith's words, constantly recalibrated what the tenant can afford to give.