IT IS A CPU LIMITATION, YOU DOLT!

[strredwolf]

A particular "software analyst" claims that the 4 gig RAM limitation in 32-bits is a joke, and says it's a Vista licensing scam from Microsoft.  http://www.geoffchappell.com/viewer.htm?doc=notes/windows/license/memory.htm for the full article.

Let me call bull on this one.

First, and most simple reason:  Some CPU's have 32 bit wide address spaces.  The newest one to have this is the Intel Atom (the one geared for netbooks).  If the address space is set for 32 bits, that means it can only address 2^32 (that's 2 to the 32nd power) bytes of RAM.  2^32 is 4 gig (not 4 billion bytes, that's the base 10 version).   To get even more RAM address space, you must have a 64-bit processor that can address more space... and not all of them will do it.  Some Core 2 Duo's are stuck in 32-bit hell, but AMD's Athlon 64's, 64 X2's, and Phenoms are all 64-bit.

Now, to get even more technical, because there's a few "gotchas".

Second:  Because a 32-bit CPU can only address 4 gigs of space, and a video card (like the Nvidia Geforce cards) has up to a gig of RAM for itself, you're going to loose whatever main memory you have if you max the RAM for the CPU out to 4 gig.

Example:  Nvidia Geforce 9800 on a WinXP system, having 4 gigs of RAM and a 1/2 gig on the video card.  XP will report at most 3.5 gigs because the video card has to be fully seen.

You don't have that limitation in 64-bit mode.

Third:  There is a way around it, but it's crude and comes from the 16-bit days of the IBM XT.   Back then, it was called LIM EMS (Logitech/Intel/Microsoft Epanded Memory Specification), which swapped 64K chunks of RAM supplied on an expansion board in and out of a reserved area of the 1 Meg address space the old Intel 8088 CPU has. 

Now, in 32-bit land, a similar technology is called PAE.  And it's only useful for swap, because it's slower than accessing the RAM directly (in 64-bit mode) but faster than smacking it onto the disk.

Now who thought up of PAE (Physical Address Extention)?  Intel, back in the Pentium Pro days.  The same guys who invented LIM EMS!

Forth:  Vista's a joke by itself anyway.  Wait until October and get a copy of Windows 7, or go with a copy of Ubuntu 64-bit.

Comments

[jecook]

There's one other quirk:

running a 32 bit OS on a system that's capable of running 64 bit (i.e., WinXP Pro on a Core2Duo or other processor that's 64 bit capable) gives you the same result- you can throw over 4 GB of system memory at it, but the OS won't use it.

Other then that, total agreement: I called BS on the dude halfway through the article, and Vista is a total and complete joke. I reckon my work is going to go from XP Pro directly to Windows 7 once it's stable.

[kesarra.livejournal.com]

The only imposed Microsoft limit in regards to RAM is its regulations for XP laptops in the 7-12" range.

[strredwolf.livejournal.com]

That's only for Windows Starter edition, and it's about 1 gig for the upper limit. Vista 32-bit is limited by the CPU addressing capability, for 64-bit processors only in 32-bit mode.

XP in general is 32-bit (Yeah I know there's XP 64 bit but it's buggy)

[kazriko.livejournal.com]

Good post. The last thing we want to do is return to the bad old days of segmented memory. Flat memory model 32-bit was the best thing to happen to computer science in ages. I remember the pain of setting up overlays in Pascal so they would page in and out of the 64k pages, and how irritating it was to maintain even in that simplified environment (and it made C a nightmare with 5 different memory models to pick from...)

If only 64 bit windows didn't SUCK so much. There's just so many incompatibilities with 32 bit and 16 bit software on it, and that means many of the programs I use for my job fail to work on my home Vista 64 bit system.

[strredwolf.livejournal.com]

It's not all that better in Linux 64-bit. There's still some code that uses 32-bit integers and when int goes 64-bit, all hell breaks loose. And I hear ya with C on those 8088 -- I was using QuickC to port some AppleII programs over and I had to choose which pointers I needed because of the nice 20-bit wide address bus that was segmented.

[kazriko.livejournal.com]

Yeah, int going 64 bit will be an adventure on an order higher than the Y2K fiasco. But we do have around 20 years to deal with that particular problem.

[(Anonymous)]

As the target of your barbs, please be good enough to permit me a reply...

I don't say that the 4GB RAM limitation in Windows Vista is a joke. I do say that 32-bit Windows Vista has the code for using memory above 4GB and that the only mechanism through which this code is withheld from use is the license data.

Given that I present demonstrations of 32-bit Windows Vista using memory above 4GB, your call of "bull" on this is not just a criticism of me as being mistaken or ignorant or whatever. No, you are in effect accusing me of a hoax. Call me a dolt, as you like, but to call me a criminal is a bit much.

First, and "most simple":

What are you trying to say regarding the Intel Atom? The existence of hardware that does not do feature X tells nothing of whether any operating system supports feature X.

You are of course correct that a 32-bit processor can only address only 2^32 bytes in the address space it provides for memory references in the instruction set. But those (linear) addresses are not necessarily the (physical) addresses that go out to the data bus. If the data bus is 64 bits and a 32-bit processor is given the means to translate 32-bit linear addresses to 64-bit physical addresses, then a 32-bit processor with a 64-bit data bus can address 2^64 bytes of physical memory even while the linear address space is stuck at 2^32.

Second:

If a device has only a 32-bit bus, then yes, it needs physical memory addresses beneath 4GB. The address range that is taken for the device overrides any RAM that might otherwise have been accessible at those addresses. This RAM is lost to you unless your chipset can remap it to other physical addresses (above 4GB). Without such remapping, the memory is lost even if you have a 64-bit operating system and processor.

Third:

It seems you have found a connection between PAE and EMS. Yes, they both came from Intel.

As technologies however, they are completely unalike. PAE has nothing to do with swapping memory. With PAE enabled, a 32-bit CPU accesses memory above and below 4GB as directly as does a 64-bit CPU. Rather, both access memory _indirectly_ through page tables to translate linear addresses to physical addresses. If you find this sort of memory access satisfactory for a 64-bit operating system on a 64-bit processor, it can be no less satisfactory for a 32-bit operating system on a 32-bit processor (with its 64-bit data bus). Indeed, the translation is strikingly similar, and is actually a little simpler (and quicker) in the 32-bit case.

Forth:

Except for your spelling, I agree with you on this point. That something is observed about product X says nothing of whether product X is being recommended.

Geoff Chappell.

[strredwolf.livejournal.com]

I'm glad to see you reply! Brightens up my day.

That said, you have some good points and a good familiar misunderstanding.

First: The 64 bit data bus only carries data. That's it, nothing more. That's why it's name "data bus." There is a separate bus, called the "address bus", and on Intel processors, it's at most 48 bits. AMD? A full 64.

What does the address bus do? Simple. It indicates what address to get the data from or put it to, which is delivered or sent via the data bus. The direction is done by seperate pins, plus some administrata pins like the address bus latch (because rarely you can set tne whole address in one go).

So you can grab 64 bits of data, but can you address the entire space of the RAM physically? If the CPU physically can't address it, then PAE or other slight of hand by the OS won't work because it doesn't have the ability to actually do it! Separate 32-bit address bus, remember?

Now, you say that you can demo Vista on a 32-bit CPU addressing beyond 4 gig of ram. I'd say "You're a 64-bit processor running in 32-bit mode, and the chip has more than 32 pins on the address bus." Licensing? Check your CPU first. You'll find more when you research the chip itself.

[(Anonymous)]

Sorry, yes, change "data bus" to "address bus". You have me flummoxed as I try to grasp why you call me a dolt and (in effect) a fraud when it looks ever more that you're criticising me for something I never wrote.

You know, the article is about how 32-bit Windows Vista has the code to use PAE for access to memory above 4GB but is prohibited from using any memory above 4GB, whatever the processor, whether 32-bit or 64-bit, no matter how capable. The entire mechanism of that prohibition is licensing. Remove the mechanism and you let the code get to work on memory above 4GB.

Obviously, for that code to work successfully, the processor must support PAE, and the chipset must handle memory above 4GB, and I guess I ought also spell out that the RAM must actually be fitted!

What on earth does it matter whether the PAE-capable processor is 32-bit or 64-bit? Nowhere in the article do I go anywhere near suggesting that 32-bit Windows Vista has code that can use memory above 4GB on all 32-bit processors. Nowhere is the article invalidated if the PAE-capable processor has a 64-bit instruction set or 64-bit registers. Your criticism, if I yet have wrapped my head around it, is just of something that you have read into the article for who can know what reason.

So, yes, the article's demonstration is on a 64-bit processor, as is clear enough from the picture. What of it?

Geoff Chappell.

[strredwolf.livejournal.com]

I was in a heavy-duty rant mode there -- I couldn't believe it was a license issue instead of a technical issue. Forgive the Eastern Shore gail-size blowhardness. I also tweeted about it, and got into some other discussion too. There's some credence to the claim, but there's still a few niggling items.

The main thing I thought was that the non-64-bit processors only had a 32-line wide address bus size. Of course, that's not true for all processors; Pentium 3's have 36 lines and thus can do PAE. The max Intel and AMD has is 44 lines.

That, in of itself, lends some truth to your article. The demo unfortunately only lends a sliver of truth to it for one reason: It was on a 64-bit processor. What would stop Microsoft from running on a thin 64-bit shell and 32-bit emulation mode most of the time in Vista, thus avoiding using PAE?

That may be outlandish, but there's a nice test for it. Get a Penitum 3 or 4 system built with over 4 gigs of RAM and a chipset supporting the P3/P4's PAE abilities, and run the "unlocked" 32-bit Vista on it. If you can address all the RAM in there, then you have much more solid proof that 32-bit Vista isn't licensed for the extra RAM.

It's a good theory; proving it on the P3/P4's will turn it into fact.

[(Anonymous)]

The article is about operating systems not processors. It touches on processors only from the operating system's perspective. As long as the PAE kernel believes it is running on a processor that executes x86 instructions with PAE enabled (and with memory to match), what does it matter to the logic of my article if this execution environment is created by a "thin 64-bit shell" or by a 64-bit processor that virtualises the x86 instruction set on the chip or even by an emulator in software?

However this execution environment comes about, the only reason that the PAE kernel won't look at any memory above 4GB is licensing. The demonstration is as much proof as any demonstration can be that 32-bit Windows Vista, which does not ordinarily use memory above 4GB even with PAE enabled, will use that memory if you enable PAE and disable the two license values.

Well, perhaps it leaves open that the simple change of those license values suddenly opens up a whole new execution environment so that memory above and below 4GB suddenly gets virtualised in a way that memory below 4GB isn't ordinarily - but although that's conceivable, it is beyond incredible. Anyway, if that were happening, then surely the memory element of the Windows Experience Index, for what it's worth as a performance measure, would degrade badly when memory above 4GB is brought into use - and, no, it doesn't.

Now, having defended the demonstration as not being intended for what you seem to expect of it, I must of course agree with you that the demonstration does not prove that the PAE kernel actually is using PAE to access the 8GB. OK, so suppose I'm interested. I'd begin by asking how else might the PAE kernel be accessing the extra memory except through PAE and I'd follow with the observation that alternatives would be not just outlandish but fantastic.

If you need to prove to everyone's satisfaction that the PAE kernel actually is using PAE, then yes, you could try running the demonstration on a processor that everyone agrees is incapable of anything relevant except PAE. I'm happy for anyone to do it if they feel the need, but it seems excessive to me.

Even as someone who likes more than most to dot every I and cross every T, I'm entirely happy to presume that the PAE kernel actually does use PAE (however the PAE gets implemented). To anyone who needs it proved, I say it would be easier just to inspect the page tables with a kernel debugger and see that they conform to the PAE algorithm. Indeed, it would be easier to step through the BOOTMGR and OSLOADER with the boot debugger and watch it enable PAE and set up those page tables. If my article doesn't give you enough detail for where to look by yourself, I can easily enough tell you more.

Geoff Chappell.

[strredwolf.livejournal.com]

Well, it's hard to not get into a discussion of hardware when you're talking about a hardware feature of a CPU. Operating systems by default have to be intimate with the hardware -- Windows just isn't that intimate like Linux (depending on the distribution used) or even old OS/2.

That said, I did feel the demo on a 64-bit processor in 32-bit mode could open up too many avenues for access or non-access, and really not proved your point. There could be a different way of doing the access (native 64-bit/32-bit switching, VM emulation); there could be a limiting factor (VM again, disabling PAE; BIOS turning off PAE; 64-bit/32-bit switching can't handle it). Forcing Vista to use PAE on a Pentium 3 or -4 would do it, and is the better option than decompiling Vista.

All that said, I go all the way back to my fourth point: Why Vista? It's a joke. Ignore it -- Gartner urged businesses to go Windows 7 instead of Vista. Focus on that OS instead. If Windows 7 has the same licensing issues, then you'll have some news for Paul Thurrott over on the Supersite for Windows and the Windows Weekly podcast.

[(Anonymous)]

You can feel as much as you like that the demonstration leaves open the question of precisely how the 32-bit operating system accesses memory above 4GB. The point is that nothing like the demonstration should be possible if your first post is justified.

The article gives a perfectly reasonable and sufficiently detailed explanation of how the PAE kernel comes to be limited to using only memory below 4GB. Remove the limitation and it uses memory above 4GB too. Whether the PAE kernel actually does use PAE (as seen by the chip) is just not the article's domain and is not necessary for anything the article claims.

Whether the PAE kernel actually does use PAE (as seen from the operating system) is not proved, I accept. It's just the obvious explanation that fits the observations, and it's the only explanation that isn't fantastic. If you need to chase the fantastic to justify having called me a "dolt" and a fraud, well, I leave you to it.

On minor points...

Finding a Pentium 3 or 4 with a chipset that takes 4GB may be easier for some. Debugging may be easier for others. Disassembling may be easier for hardly anyone, but it's what I do. How else do you think I even imagined the patch except by knowing of the memory-limiting code first?

Yes, Vista is a half-baked rush release with many irritations and outright defects. But it has been for nearing 3 years the current version of the dominant operating system. Inasmuch as anything to do with computers is worth understanding, Vista must be understood, as must the next Windows version, good or bad. As an aside, consider that theology faculties all round the western world have atheists on the staff.

As for Windows 7, I haven't seen it yet - disassembly takes enough time that I work only on released versions - but numerous websites in China and Taiwan had reproduced the patch for Windows 7 within a couple of weeks of my posting a draft of the article in January. I leave it to others to decide whether it's newsworthy. I have my work cut out finding out what Windows actually does.

Geoff Chappell.