We consider the question: ``How bad can the deformation space of an object be?'' (Alternatively: ``What singularities can appear on a moduli space?'') The answer seems to be: ``Unless there is some a priori reason otherwise, the deformation space can be arbitrarily ugly.'' Hence many of the most important moduli spaces in algebraic geometry are arbitrarily singular, justifying a philosophy of Mumford.
More precisely, every singularity of finite type over Z (up to smooth parameters) appears on the Hilbert scheme of curves in projective space, and the moduli spaces of: smooth projective general-type surfaces (or higher-dimensional varieties), plane curves with nodes and cusps, stable sheaves, isolated threefold singularities, and more. The objects themselves are not pathological, and are in fact as nice as can be: the curves are smooth, the surfaces have very ample canonical bundle, the stable sheaves are torsion of rank 1, the singularities are normal and Cohen-Macaulay, etc.
Thus one can construct a smooth curve in projective space whose deformation space has any given number of components, each with any given singularity type, with any given non-reduced behavior along various associated subschemes. Similarly one can give a surface over Fp that lifts to p7 but not p8. (Of course the results hold in the holomorphic category as well.)