This article presents a dynamic view of how institutions external to the state foster informal relationships that influence the voting preferences of elected officials. Institutions external to the state can activate different identities, potentially leading to outcomes at odds with predictions based on party identification and political ideology. This theoretical perspective is applied to a period of increasing partisanship and polarization—the birth of the second party system in America—when congressmen lived together in boardinghouses chosen independently of preexisting political preferences. Common residency is found to mute the effect of partisanship while reinforcing the North-South divide; this effect was strongest during the period of most acute polarization. This finding sheds light on the interaction between state and nonstate institutions and their differential impact on the political process. It also suggests a cause of current high levels of political polarization: the lack of time congressmen spend together informally.