The trouble with conceiving of politics as solely or predominantly an arena for the assertion and ratification of rights is that it inevitably favors the powerful over the less powerful.
A "right" in its simplest etymological and legal form (ius) is simply a claim of ownership over something (or someone). If this is my piece of pie, I have the "right" to eat it--or throw it away. But then, of course, the question quickly presents itself of just why I have this right, and how it relates to the rights or potential rights of other people. My friend feels a strong desire to eat this same piece of pie; does he have the right to do this? Do I have to respect his right by giving him my piece of pie? As in this example, the assertion of different rights in the real world quite often involves simply logical contradiction--while in many other cases, it merely involves obvious conflict, as there would be if I asserted my right to eat the pie, and my friend asserted his right to use it (temporarily) as a doorstop. Etymologically and philosophically, the only way to adjudicate these sorts of disputes is by reference to justice, which is simply the abstract universal form (iustitia) governing the concrete particular act of claiming or possessing a right (ius). All politics quickly reduces to ethical claims about justice--that is, who has a right to what according to our overall philosophical system, our understanding of the good, private and common, retribution, distribution, etc. This is not necessarily easy in each practical instance, nor is it intended to be.
Contemporary politics is largely an attempt to have rights without justice--and implicit in this, or often explicit, is the idea that rights ultimately depend, not on a rational, universal form of justice, but simply on law conceived of as a purely positive assertion of the State or some purely formal legal or contractual procedure. Yet if politics is purely about asserting rights and getting those rights ratified by the State or some other procedural machinery, it seems fairly obvious that the rights that will in fact be ratified will be those of the people who in fact possess the political power or expertise in manipulating contractual or other procedural machinery to have their rights ratified and those of others denied. If the definitions so far are correct, this claim is more or less tautological.
More fundamentally, the use of force to assert a right to something (or someone) to which one has no actual right is the most straightforward definition of theft (and violence) there is. Yet if a right is nothing other than a claim on something or someone, and if the validity of each and every right ultimately derives merely from the fiat of the State or another purely formal machinery, it's difficult to see how this does not, in the end, simply reduce right to force, or at least a sublimated or redirected form of force.
Under such a system, I assert "my" rights, and am met in return with my neighbor's assertion of his conflicting or even contradictory rights. What adjudicates between our two claims is not, in fact, any universal rational principle, but merely the relative power or ability of us both in relation to political power and/or the machinery of legal or contractual power and enforcement.
If I steal my neighbor's farm by force, and then have that theft ratified by a legal authority, have I violated any principle of politics? What if I passed the law first, and then stole the farm? This is not a fanciful example, but is in fact a straightforward description of the English Enclosure Laws. If I kill my neighbor, but then justify it by the claim that he was violating my right to privacy (or property, or autonomy, or security, or subsistence, or self-actualization, or national identity, or), and have that legally ratified, have I committed violence? How would we know?
It should be straightforwardly obvious that the rights in turn asserted by different groups and different "sides" in our current political discourse are in conflict or even contradictory to one another. The question, then, is whether it is possible, absent a robust philosophical idea of the good and justice, for these claims to be adjudicated in any way that does not ultimately reduce to force. Another question is to what degree it is possible for our political system, inasmuch as it is founded on purely formal adjudication of contradictory claims, to continue to successfully sublimate or redirect the direct, private use of force among these different parties.
In any event, though, my objection to modern rights discourse is not just practical, but more basic and philosophical. A political system that consists predominantly of the assertion & counter-assertion of rights & their purely formal ratification by the state or other procedural machinery is inevitably going to be one in which theft and violence (philosophically speaking) are both encouraged and frequently approved, and where the powerful dominate the powerless with little fear of retribution. This is my fundamental objection.