Joseph Harmer, ULC Minister