Making QT and MSVC 2K8 dance, part 2

So the last time I ran thru pretty much everything you needed to get QT built against the MSVC systems. However that article didn't include anything on how to actually get QT to play nice with visual studio in the context of project management and compilation across multiple platforms. In the pre-QT 4.5 world this was allowed only by those that had paid the (which are completely ridiculous, even if your an academic institution) commercial licensing fees. Since then, a new project has started and become available on the qtsoftware.com site called 'vsaddin'. This plug in made it's 1.0.0 debut late last month and I've gotten a few chances to work with it.

Before I continue on with how to use this addin, I'll give you the link for those that are just looking for that: VSAddin

On to the main discussion, the vsaddin doesn't do everything for you, this simply isn't possible. Visual Studio and the QT system think distinctly differently, so it can't be assumed that it will do everything, but it will make life much much easier.

There isn't much to do for 'configuring' the VSaddin program. Really you just shut down visual studio, install the addin and reboot. Once you've done that you have a new button on your toolbar simply labeled 'QT'. This menu will give you access to a variety of functions useful for different angles of development, like quick lauch buttons for QT Designer and QT Linguist. It also includes a few project import/export options and last but not least the QT Project Settings (grayed out if your not in a QT project) and the QT Options button. QT Options is where you want to be for general configuration, in here you can add pointers and names for each compiled copy of QT on your system.



As you can see, you can include connections to both standard Windows and Win CE versions of QT builds. I'm not sure what the difference is, but I don't develop WinCE applications. That said it, if you know how to compile one of those (and know what the difference between a standard compile and a WinCE compile is), you can use them as well. The other tab 'QT Default Settings' allows you to modify the relative paths of things like the 'Generated Files' (ie, files generated by the QT framework for your sanity's sake) and controlling what options are delivered to MOC by default.

The rest of the program integrates into the Visual Studio tool chain directly, showing up as a project selection when you make new projects. That QT Project Settings allows you to among other things change the version of QT your currently using to compile. My big complaint here is you can't set separate ones for x64 and x32 instructions and have to go in and change it manually.



Once you start a new project using the QT new project the magic starts happening. The big differences are the access to .ui files as part of the project, access to a few key words (signals and slots show up in nice blue text), and a folder of 'Generated Files' that allows you to review on the fly generated files that have come up as a result of the files you've added and manipulated in your project. I'm unclear what the ramifications of these files are for things like AnkhSVN as my current QT work is un SVN'd basic programs, but I'll find out soon.

Another mark of primary interest is the ability to import QT projects into the VS environment and manage them directly, avoiding makefile based project management which avoids the annoying side effects that break things like intellisense. This allows you to build these projects using the 'build' or 'debug' commands inside visual studio without doing anything special including MOCing files for you. The 1.0 version came out recently and seems to be pretty well rounded. I have gotten a previous version a bit confused with some files that I had the plugin MOC and compile then gone back and modified heavily. This would result in an error indicating the object doesn't exist or something of that nature, running a project 'clean' then compiling worked. This solution is messy but effective if you have any problems.

It also adds integration for the .ui file format into Visual Studio without alot of finagling. It still loads the QT UI editor, but they are now clearly labeled and double clicking them instantly boots the editor with the appropriate dialog / menu displayed. While this isn't 'perfect' UI integration, I'm not convinced perfect integration is worth it. There are semantics and thought processes in QT that don't necessarily make sense inside the visual studio drag and drop editor. So isolation is helpful in my view, as the domain specific application for gui development is far more feature complete than a kludge to make visual studio integration 'perfect'.

As with all things, the project integration isn't perfect. While you can import a project with little difficulty and almost no issues (from the few tests I've done), the project export certainly isn't everything one would want to see. Mostly the project export consists of building a series of template qmake files with all the basic information filled in, but it doesn't provide any rules for adding custom segments to the make files. For instance if you want to use CUDA with a QT project you could, but you'd have to go into the relevant make files for all files that use CUDA and add the proper script (or add it to the head file, depending on how your implementing your project).

The Blue Pill:
The long and the short is, version 1.0 of a pretty solid QT - Visual Studio integration tool is out. It's rough around the edges but generally gets the job done. For anyone fully familiar with QT development that just wants to take advantage of what Visual Studio has to offer this is a huge help. If your just starting out with QT and you know visual studio this will also be a big help for getting projects written and running under Visual Studio but will require a learning curve to get those projects published across QT's variety of target-able platforms.

The Red Pill:
That said there's a few things that hard core developers will probably gripe about, the lack of ability to add preset custom rules to project files is the big one I can think of. And Microsoft purists I'm sure will be offended that the system doesn't offer full UI design integration, but as I stated above I don't consider this a bad thing. A few of the errors the system throws when it encounters a problem MOCing or Compiling could use some improvement as well as they suffer from 'mystery error syndrome', a problem becoming all the more painful these days. Also I get some weird errors building x64 code against x64 copies of QT, that seem to indicate it's belief that the QT version I've selected doesn't exist (and is a lie, kinda of like cake); however it still happily compiles using said library. It would be nice to be able to specify separate library paths for x64 and x32 versions of QT so that you were always compiling against the correct one, but it takes all of a second to change in the project menu. All that said, it works, which is better than the end of my last QT post. Back then it didn't exist.

Decoding the 10 Gb Switching World

I've had the pleasure and frustration over the past month or so to read up and explore a number of different switching systems available for high performance computing. Specifically in the realm of 10 GbE connectivity. It's finally winding down now as we finally begin to start filling out our PO's and having our shiny new computers so I thought it was an appropriate time to share the funness I've had to absorb in learning about this world of what can be called anything but 'standard' for Ethernet connectivity.

I may seem a bit facetious to say that there's something non standard about anything involving the word 'Ethernet' but in the 10 Gb realm it seems the standard suffers somewhat from flavor of the month syndrome. To qualify that there are in the market now a number of competing standards that involve a variety of both copper and fiber solutions.

There's a few different types of 10 GbE available these days. General copper connections, fibers connections and a connection format known as SFP+ that can use both copper and fiber.

The copper connections typically boil down to CX-4 an infiniban 'like' connection that's generally exclusive to itself. While it's prevalent throughout the industry it appears to be rather exclusive to it self, as very few of the switches allow for connectivity to other types of 10 GbE at all, and those that do typically provide very few connections for other types of connectivity. An alternative that will allow for a smoother upgrade path for existing architectures is on the horizon in 10GBaseT and is compatible with earlier BaseT links in some cases but not all. This is dependant on the switches ability to step down, the one we've been working with has a standard 1GBaseT link that can not step down to a 10/100 connection. The good thing about these copper solutions is that they are relatively cheep (compared to a fiber system) and generally well understood. They are also more robust than a simple fiber connections making them more preferrable in a rough environment.

The second vien of connectivity is the general fiber connectivity. These fiber systems in the 10 GbE realm can be utilized for both long distance communication over single mode fibers for connectivity of up to 10-40 km depending on the lasers used (fibers in the range of 8 micro meters thick) and shorter distance multi mode fibers (in the 50 - 65 micro meter thickness, 50 being the newer more versatile cables). It should be noted that there are long range multi mode fibers but their 'long range' is over 2 and a half orders of magnitude shorter than a single mode fiber. Regardless of the details involved the big killer for this technology is how expensive the module connector construction is coupled with the cost of the cable and it's termination. These connectors are expensive mostly because they recieve the packets of data from the switch itself and must generate an analog signal of that packet then run it thru a conversion to make it into a light transmission, these modules also include all of the hardware required to make the process go in reverse as well making individual port price relatively expensive for many of these systems.

In the above systems for 10 GbE connectivity I've left what is the latest implementation of the standard known as SFP+. This was left out because it spans both copper and fiber connectivity in the same modules instead of being exclusive to a given type of connectivity. It's based off the same technology as fiberchannel switches with the major differences being that instead of each module having to do the conversion to a 'transmission' signal, the switches generate those signals. The modules just implement the transmission to light conversion and vice versa. This modular system allows for the creation of copper connectivity thru cables know as 'twin-ax' cabling, where the 'connector' instead of being a module with a jack in it that an infiniban esc cable connects to is simply built onto the end of the cable. This allows for a far more robust cable connection than the connectors you would see in a CX-4 / infiniban / BaseT link, however the twin-ax specification also has a severely limited range on the order of 5-10 meters (many systems are not certified for 10 meters and max out at 5). In contrast the fiber connections are modules with fiber plugs available for connecting to the standard fiber cables of their particular specification (including SR, SRL, LR, and LRM connectivity based on the device support I've seen recently). In my view, the best thing about this solution is that you don't need a bladed switch with different connectivity frameworks to enable you to transmit and receive on different fiber and copper mediums by simply changing the module in a given slot.

The Blue Pill:
While there are a myriad of different choices out there the trick seems to be picking a back end and sticking with it. Going for CX-4 on your servers? Great, make sure your storage vendors understand they will be obligated to give you a CX-4 connection (assuming you have a desire for a flat network). The good thing is many of these technologies are already supported by companies, and if your using a third party vendor, they can typically get you a card that will work and is certified for your servers. We recently acquired a sun system with an SPF+ card from Chelsio due to the fact that we have standardized on the SFP+ network architecture and sun doesn't produce 10GbE connectors for SFP+ itself. Our storage cluster also uses Chelsio cards to provide SFP+ based fiber connectivity from our high speed disk to both the world and our computation cluster.

The Red Pill:
The big problem isn't typicaly finding options, it's using the options you have at your disposal. If your company has a deal with HP, your getting HP servers, and depending on your restrictions you may be getting HP switches as well. Knowing that you had better make sure your storage vendor can connect the the 10 GbE system your using. The switch vendors we have been dealing with have told use that the fiber connections utilized by some other systems (XFP for instance) will work with the appropriate SFP+ modules but the last thing you need is something going awry because your mixing connection types in this way. As such we've decided that for our purposes we will skip using alternative optical media connectors when we can use an SFP+ system to guarantee we are always using the same interfaces thou they do use the same technical specification for their transmission (that said our uplink to the rest of the world is not in fact an SFP+ link on the other end).

TWC expects broadband stimulus, without stimulus strings attached

Time Warner Cable (TWC from this point forward) sent a letter to the FCC earl yesterday stating:

"Now is not the time, nor is this the appropriate proceeding, to engage in a debate about the need for net neutrality obligations. The clear, overarching purpose of the ARRA (American Recovery and Reinvestment Act) is to jump start the deployment of broadband facilities in unserved areas and to expand broadband availability generally, thereby creating jobs immediately and extending the near-term economic, educational, social, and other benefits of broadband services. Debates in this proceeding about new net neutrality regulations would only divert attention from these important goals, delaying the distribution of funds while generating considerable contention when the Commission should instead be fostering a spirit of collaboration. That delay would be exacerbated by the need to comply with the requirements of the Administrative Procedure Act in connection with the adoption of any new nondiscrimination rules."

This is all well and good, but the idea of the stimulus money (in my mind, and I may be delusional; hey you never know), is to help create a network for the people. As it happens it's a network WE are paying for. Stimulus money doesn't come from a sock under Obama's bed, it comes from tax payers. In consideration of that, should we be forced to pay for an infrastructure used to abuse it's customers?

For those wondering if I'm baselessly accusing TWC of abusing it's customers or not, take a gander at a few of the latest articles from around the tech world concerning TWC. Here's a few quick ones from Ars, Wired and The Business Insider. Basically the math works out like this. If you are charging some one for both the speed at which they access the data and the amount of data they can use, except in cases where that data comes from the ISP you can guarantee your loyal customers are happy with their low rates. Where as your disloyal customers using other vendors solutions on your network pay a premium as they chew away at their bandwidth caps and are dinged to raise their cap for that period of time. This effectively allows them to punish heavy users of services that compete with their own in a way other than Comcast's earlier attempts to manage the 'problem' by throttling competing businesses. Different thou it may be, it gets to the same ending, abusing what is effectively a monopoly position because your consumers have no choice but to acquiesce to your demands. On top of this it allows you to abuse your competitors by simply making them less appealing than your own offerings, not by reasonable competition but by effectively taxing their services. It wouldn't surprise me if this is treated as anti-competitive practices instead of even worrying about the net neutrality issues, your allowed to compete in this country but sabotaging your competition is going a few steps to far.

These caps may not be so painful if they were at least reasonable sizes. Comcast does in theory cap their bandwidth but it's a number I have never succeeded in hitting (it's in the 30-250Gb range). TWC starts at 1 Gb. If your using their lowest tier and streaming an HD Hulu movie, you will cross that limit in an hour and twenty minutes (or there abouts) meaning you can't even watch an entire movie without getting dinged for more bandwidth. Whereas you can watch as many on demand movies from TWC's collection as you would like without impacting your bandwidth cap, assuming your impressed with their system (I don't have road runner, so I can't speak to it's quality first hand; but it was a much derided service during my time in Philadelphia).

Regardless of the drive, the whole thing is rather warped by the fact that we the people will be the ones paying for this us, the tax payers. What I'm speaking of specifically is the so called 'Broadband Stimulus Bill' meant to provide money for building broadband infrastructure in areas that are either not served or are underserved. One major danger with allowing behavior like this to go forward in these areas is that these users may never know anything else, in all likelyhood a single provider will develop a broadband pressence for a given under / unserved user base. In these cases especially, it isn't unreasonable to demand a respectable level of net neutrality as we would effectively be gifting these companies a monopoly pressence in these regions. Making the TWC stance of give us money but don't even think about trying to put those Net Neutrality strings on 'our' money stance all the more disconcerting.

The Blue Pill:
The upside is that these plans have caused quite the uproar both on the blog world and in the public sphere leading to one Congressman beginning the process of writing a bill to ban the bandwidth caps completely. This however for the time being will now not be necessary as TWC has announced this evening that they will shelve the plans while 'customer education process continues' (as noted on Ars). So for now it's a bit moot, but in the near future may rear it's head again.

The Red Pill:
The line '...while the customer education process continues...' indicates that while we don't have to worry about these issues for the next week and probably the next few months this will not be the last time we see this. Remember that Comcast does in fact have a usage caps but they are high enough that even heavier users are a hard time hitting them. While that may not continue as the web gets steadily more complex requiring signifigantly more data transfer power if the caps keep up with the complexity it won't cause any huge problems (I still don't like it, but I doubt I'll hit it any time soon). Regardless, among those I know your ISP is the least loved of the bills you pay so making your image worse doesn't seem to be a good idea.