Presumably you mean to ask about well-formed, not valid, XML.  (See Well-formed vs Valid XML for details on the difference.)
Newline characters are most certainly allowed in well-formed XML.

 (#xA) is CR 

 (#xD) is LF 
(Windows usually end lines with CR+LF; MacOS X and Linux, LF; classic Mac OS, CR.)
The XML Recommendation does indeed clearly allow both.  See Character Range:
Char     ::=      #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] |
[#x10000-#x10FFFF]
Common Usage
Within an element, new lines are typically significant to an application:
<a>one
two</a>
usually means something different than
<a>one two</a>
Between markup, new lines typically are insignificant:
<a>
   <b>one</b>
</a>
usually means the same as
<a><b>one</b></a>
Other Characters
Finally, you're painting in somewhat sloppy strokes in saying that &, <, and > are illegal.  Instead, use the following guidelines:
&: must use & if not a part of an entity reference. 
<: must use < if not a part of a tag, comment, PI, etc. 
>: must use > if part of the string ]]>. 
': must use apos; if within attribute values delimited by '. 
": must use quot; if within attribute values delimited by ". 
See also