And then you get things like HP calculators.
Let's see... the 41C and 1xC families had the most advanced variant of the original processor design... 56-bit registers, 16-bit addressing, 10-bit instruction words (although, before the Saturn processor used in the graphers among other things, HP calculators were Harvard architecture - Saturn switched to a von Neumann architecture), supporting 8-bit chars for some ASCII work, optimized for 4-bit nibbles of BCD data, but being a bit-serial processor inherently. (And, the original implementation, in the HP-35/45/55/65 and 70/80 was 56-bit registers, 8-bit addressing, 10-bit instruction words, no 8-bit support, 4-bit BCD optimization, and bit-serial inherently.)
In any case, the bitness refers to how many bits make up the inherent smallest unit that the computer understands. That said, a 32-bit word that is the smallest addressable unit isn't horribly weird - I mean, most modern processors are doing 32 or more bit loads anyway, and you just do operations on that 32 or more bit register to get the 8 you want. A byte is normally 8 bits. Different word sizes depending on instructions or data is much weirder, especially if it's a von Neumann architecture machine.
Guess I need to explain Harvard vs. von Neumann architecture. Harvard architecture, the machine has two completely different pathways for programs and data. This isn't really used much nowadays at a whole system level, but elements of it are used here and there. von Neumann architecture, you have a bunch of memory, and programs and data can be anywhere in it.