This is joint work with John. The Weil representation is a well-known tool to study arithmetic and cohomological aspects of orthogonal groups. We construct certain, "special", cohomology classes for orthogonal groups O(p,q) with coefficients in a finite dimensional representation and discuss their automorphic and geometric properties. In particular, these classes are generalizations of previous work of Kudla and Millson and give rise to Poincare dual forms for certain, "special", cycles with non-trivial coefficients in arithmetic quotients of the associated symmetric space for the orthogonal group. Furthermore, we determine the behavior of these classes at the boundary of the Borel-Serre compactification of the associated locally symmetric space. As a consequence we are able to obtain new non-vanishing results for the special cycles.