There is no reason to view emotional or mental deficiencies as different in kind from physical ones. To put it in the harshest of terms, if you think someone who is born blind is deserving of sympathy and support then you should think someone who is born lazy and stupid is deserving of sympathy and support.
Further once you concede that the lazy and stupid are deserving of sympathy then its difficult to construct a set of poor people who are not, since these are among the least sympathetic qualities that could cause someone to be poor.
Thus the vast majority of the poor are deserving of sympathy or support.Laziness is not an unvarying or testable condition, like blindness or stupidity, so it is possible to fake it. In other words, rather than being lazy in the congenital sense (for which you are not to be blamed), you might simply be inconsiderate (others can do the work while I sit under this tree). Smith notes of the former that "we lack the technology to distinguish them from those faking laziness", so any attempt to discriminate between the two will fail.
Smith is a neoliberal (in the spirit of his namesake), so would be happy were we to invent technology that could accurately distinguish the fakers from the helplessly lazy, but purely for reasons of efficiency rather than morality. His proposal is provocative because it eschews morality.
Smith recognises that motives, and thus culpability, are not distinguishable when it comes to a person's lot. As well as circumstances beyond their control, we have to recognise that their choices are relative and subjective. There is no common scale that we can measure everyone against, thus we can never say that someone deserves what they got. Maybe they did, maybe they didn't. It's a matter of opinion, not empirical fact.
Essentially, this is an argument for excluding morality ("just deserts") when discussing people's lot in life. This is attractive from a practical perspective. Morality is messy, costly to get agreement on and mistakes will be made. It is also helpful when applied at both ends of the spectrum. The better criticism of CEO pay and banker bonuses is that they indicate a systemic failure (inadequate regulation, a rigged market, the agency problem) rather than a moral shortcoming (individual greed).
However, there is a danger in taking this approach. The neolibreal view holds that "fair" is whatever the market decides, and is therefore essentially amoral. It also holds that individuals make rational choices to maximise their utility, hence choosing to live on benefits may be a perfectly rational preference. It is from this observation that neoliberals conclude that benefits must be low enough to be unattractive as a rational choice, i.e. the 'less eligibility' principle of the workhouse.
The logical sleight of hand in the conclusion rests on the assumption that working is better than not working, so the former should be encouraged and the latter discouraged. This view is also found on the left, e.g. "he who does not work, neither shall he eat" (though it's worth noting that the famous use of this phrase by Lenin was directed at the parasitical bourgeoisie, not the lazy lumpen proles). That said, there remains a strong tradition of anti-work on the left as well (I'd just like to say that I'm doing my bit).
The proposed £26k cap on benefits may sound like an attack on an exceptional anomaly, but it is significant that the rationale focuses on the absurdity (the unfairness) of benefits exceeding most workers' income, rather than a quirk in the maths. This moralistic argument, together with the claim that over-generous benefits dissuade people from working ("they" are inherently lazy), logically leads to the conclusion that even the deserving unemployed and disabled should not be better off than the lowest paid in work, i.e. our old friend 'less eligibility'.
Unfortunately, this merely serves to make the poor poorer for the simple reason that the minimum wage is inadequate. The report this week on pay rates among the big four supermarkets highlights a consequence of this. Increasing numbers of workers require in-work benefits (tax credits, child benefit, housing benefit etc) to achieve a living income.
The systemic failure here is that the state (i.e. the taxpayer) is subsidising private business, and privileging low-pay sectors to boot. One could argue that this is the quid pro quo for low prices at the till (leaving the question of profit margins to one side for a moment), though the logical argument for a neoliberal should be that the decision on expenditure is one for the individual consumer to make, not the state - i.e. if food were more expensive, one can choose what to buy and where, but one cannot choose to withhold that portion of tax that ends up subsidising supermarket staff.
Coming back to profit margins, the irony is that low-paid staff in Tesco and Sainsbury tend to shop at Aldi and Lidl because they are cheaper, despite having company discount cards. (See page 39 of the above report). This inconvenient fact gives the lie to claims that the supermarkets must push down on wages to remain competitive. The income subsidy by the taxpayer is thus transformed into corporate profit.
What seems to have emerged in the UK supermarket sector (and elsewhere) is an updated version of the Speenhamland system. This fell into disrepute because it interfered with the operation of supply and demand, i.e. unscrupulous employers could push down wages and thus shift greater cost onto the parish. The consequence was the Whig Poor Law Amendment Act of 1834, which gave us the Victorian workhouse.
Have we advanced sufficiently as a society to consider the obvious solution, which is to provide everyone with an unconditional basic income, with other non-universal support (disability, housing) being determined on needs and means? While the debate continues to be framed in moralistic terms, the answer has to be no.