There are a two major security issues:
1. The unauthorized use of the PSTN to circumvent your LAN based Internet security policy
2. The ability of the PC to be connected to by an external host, thus providing the means to bypass your firewalls and get access to your internal network
In the first, the problem is that the user can simply dial up an ISP if they want to access content that your organization has decided to block. This can provide a means for viruses, worms and other malicious traffic to enter your network without your knowledge. In the second, this allows an external attacker to potentially "call" the computer and if it is running any kind of remote access software (for example, PC Anywhere) they could potentially gain access to the computer and thus have a machine on the internal network with which to launch their attacks.
In general, I recommend disallowing all PSTN, dial-up and analog access from the internal network. In the event that you have to provide such access though, I would ensure the following:
This was first published in June 2005