The River Chess passes below Latimer House, a mansion on the hill on the edge of the village, was once a home of members of the Cavendish family who became the barons Chesham. The Third Baron Chesham was a commander in the Boer War. The original Elizabethan house where Charles I was imprisoned in 1647 and Charles II took refuge before fleeing abroad was gutted by fire in the 1830s and the present red brick Tudor style mansion was built in 1863. It was visited by Gladstone and Disraeli and was the home of the British military's Joint Service Defence College during the twentieth century. During World War II, it was one of three stately homes where captured German U-boat submarine crews and Luftwaffe pilots were initially held before being transferred to conventional POW camps. It is now a conference centre and conference facilities and accommodation buildings have been constructed in the grounds.