So, Allah (SWT) revealed the following verse, إِلاَّ مَنْ...
That is because the only important thing that matters is the obedience of Allah's orders, not what our minds and traditions accept or not. This way, we will be able to interpret many of the events/incidents that occurred without conflicting with the meaning of ‘Ismah. For example, Allah (SWT) ordered Prophet Ibraheem (AS) to deceive his people by claiming to be sic, in order for him to have an opportunity to demolish their idols after their departure.
Here, the appearance of his act seems to be a lie and therefore a sin. However, its reality is in abidance and obedience to the order of Allah (SWT) to him, which is what matters most. Prophet Yusuf (AS) falsely accused his brothers of stealing the drinking cup of the king, and this, in the eyes of any person, is considered to be a sin.
However, Allah (SWT) ordered him to do so and He (SWT) said in the Qur’an, كَذَٰلِكَ كِدْنَا لِيُوسُفَ "Kadhaleka kedna li Yusuf" “… Thus We did plan for Yusuf…” [12:76]. Yusuf (AS) is purified and protected from disobeying Allah (SWT), even if his act seems wrong in front of the common eye. Once again, what matters most is the absolute obedience to the orders of Allah (SWT).
Hence, there is a big difference between the orders of Allah (SWT), and the typical connotations of actions even though they may coincide in most cases. We must realize that the orders of Allah (SWT) is the most important, not the act in itself. And the examples in this regards are many. Eating is permitted, however during the days in the month of Ramadan, it is forbidden.
Killing a human life is forbidden, however, Allah (SWT) orders us to use the Law of Equality in certain cases and He permitted us to kill His enemies. Sexual relations with the opposite gender is forbidden except with marriage or with the slaves whom your right hand possesses…and so on and so forth. We see actions that are permitted in some cases and forbidden in others. And in certain times, they're allowed while in others, they're prohibited.