The actor had been performing Frank Sinatra numbers and standards with the house pianist in the remote restaurant he frequented when he discovered Joel had sailed his boat into port and set about befriending the Piano Man.
Slater turned island tour guide for Joel and his girlfriend and the musician showed his new pal around his luxury yacht.
But they really gelled at the restaurant after a ballsy Billy Joel fan challenged the rocker to play for diners.
Slater recalls, "We're smoking the cigars and relaxing and hanging out and this lady joins us and she starts saying to this dude, 'Hey, why don't you get up and play piano, come on'. And I say, 'Hey lady, look, we're here, he doesn't wanna play piano. He doesn't wanna do that.'
"She says, 'I bet he doesn't even know how to play any more,' and right at that moment this guy stands up, he goes over to the piano and he starts, like, playing Beethoven and Mozart and he's warming up his fingers.
"Then he says, 'OK, Slater, show me what you got.' I said, 'Well, I kinda know a little Sinatra,' and he starts playing 'That's Life' and I start singing and it went over well. He played 'One For My Baby,' which went over very, very well.
"Then his girlfriend said, 'Come on, everybody wants to hear you...' It was Billy Joel."
Recalling the story on Jimmy Kimmel Live! on Tuesday night, Slater refused to give out the name of his secluded holiday paradise, but he was full of praise for his new pal.
He said, "He's the sweetest, nicest guy... That was one of the greatest moments of my life."
Slater was on the show promoting his new drama Breaking In, which airs on Fox.