Page 67 - eBook_Proceedings of the International Conference on Digital Manufacturing V2

Fabrication and Characterization of a Low-Cost Piezoelectric using Rochelle
Salt for Energy Harvesting and Sensor Applications

introduces two efficient attention mechanisms: Window-based
MSA (W-MSA) and Shifted Window MSA (SW-MSA).

These reduce the computational complexity while maintaining
the ability to model local and global dependencies.

Figure 22: Swin Transformer architecture for cervical
cancer classification

In the W-MSA approach, the input feature z ∈ R L×D is first
l−1
divided into non-overlapping windows, each of size M × M
patches (set to seven by default). Self-attention is then computed
within each window as equation 3 and 4:

� l
l−1
l-1
= W - MSA (LN (Z )) + Z (3)

� l
l
l
Z = MLP (LN (zˆ ) + (4)

While W-MSA is efficient, it lacks interaction between
different windows. To compensate, SW-MSA is applied in
alternating layers. This involves cyclically shifting the windows
before computing attention, thus introducing cross-window
interactions without significant computational overhead as
equation 5 and 6:

� l +1 l l
= SW - MSA (LN (Z )) + Z (5)

51