Apart from previous references, unfortunately I can't offer any practical experience as I am not familiar with that RSP. Safest path for you to follow would be to contact their Help Desk and ask them to guide you through the process of connecting the F@ST5366 TN. They may have experience with the Optus modem.
The help desk telephone number is listed here along with the procedures Flip recommend - their WAN setting for Encapsulation is PPPoE, other NBN RSPs use IPoE.
The last version of the firmware that I saw listed on the forum was SG4Y1T000085. You are not able to upgrade this as a user, it is controlled by Optus and occurs automatically - the modem regularly checks with Optus Servers for status check. I am not familiar with the F@ST5366 Menu structure. Generally, the settings for all External link services are found under a WAN or an Advanced Setup folio.
If the Optus modem works well when connected to your current modem via a WAN to LAN cable connection, you should be OK to use most of the F@ST 5366 features as a secondary network or a single network in an Access Point configuration if connected as a LAN to LAN link. Puzzling why the alternative direct connect with telephone cable from the DSL port to your wall socket doesn't work. May imply the modem is locked to Optus ?
Suggest you do contact your RSP for any technical assistance they may provide - they are OK on users having third party modems. Once your contract is up for renewal with your current RSP, you may wish to consider migrating to Optus for a single network solution.