Any X-related code that you want to put in your .emacs file should be enclosed in the if window-system-version stuff to make sure that it operates correctly even when dialing in, like this:
(if window-system-version
(progn
(require 'x-mouse)
X-RELATED STUFF GOES HERE))
For example:
(if window-system-version
(progn
(require 'x-mouse)
(setq x-paste-move-point t)))