This is a Practical Peripherals 2400SA, right? Little white plastic modem with a sloped front with modem-blinkenlights?

I think this is the kind of modem that taught me that when they label one jack "line" and the other jack "phone" they may mean it. When I got it wrong the modem behaved as above: it would go off-hook and dial but not connect. Except that they didn't label the jacks on the modem, you had to look in the manual if you wanted to know which was which. But there's only two ways to do it, so put the cable that goes to the wall jack in the *other* modem jack and see if that makes it work.