Once you are happy with your greeting, you need to upload it.

On the phone, go to the Phone application and tap on 'Voicemail', then tap on 'Greeting' and 'Custom'.
Tap on 'Record', wait a second or two, then tap on 'Stop'.

This creates a 'Greeting.amr' file and tells the phone that you may want to upload it.