Temple is 3-7 this year, and had two straight losing seasons under Drayton before this.
The Owls under Drayton had an overall record of 9-25.
They were 4-18 in conference play.
Drayton left assistant coaching roles at the University of Texas to come to Temple.
Defensive Coordinator Everett Withers has been named the interim head coach for the remainder of the season.