The idea of xG is to compensate for these variations to create the most accurate prediction they can. When analysts design their models they’re actively looking for these sorts of things. A more simplistic model might just take into account shots taken and conversion, but a complex model could incorporate an individual’s stats and starting XIs, match events like subs and cards, shot location, crucial pass type and distance, and any number of other recorded stats weighed against the oppositions defensive statistics.
Whether any model has or will get close to being reliable I have no idea!