Put simply, the card belongs to YOU and should not be used by anyone else be it your wife, your mother, your dog or your goldfish.
Once the card is used by anyone except you, it is considered your fault as you did not ensure the card is only used by you. Doesnt matter whether you told your wife to use your card or your wife took your card to use. Its still considered unauthorised usage. Likewise, doesnt matter whether u use the card to sign for purchases, to withdraw money, to credit link points or to hao lian.
Your card used by someone else is considered your fault. If someone stole your card or wallet at least u can make police report. After all, who likes to be robbed. But in this case you GAVE a non-transferable card to your wife and she lost it. Banks consider it your fault. You are lucky they dont revoke your card for abusing it. Cardholder agreement clearly says NON TRANSFERABLE.