Netscaler & PCI

Posted by citrixgeek1 on

PCI compliance.  HIPAA compliance.  X compliance.  Y compliance.  Compliance this, compliance that, it’s all the rage these days, right?  What’s a poor Citrix engineer to do?  We aren’t security gurus, we are virtualization guys.  Right?

Well, when the devices we stand up are internet-facing, and they fail a PCI scan, something must be done to correct this situation.  So, I present to you the configuration steps that I recently took to change a PCI scan from failing to passing.  Please note that this is by no means a comprehensive list, individual scan vendors may perform different scans that require other modifications, so you may have additional work to do.  But, these steps should still be considered part of a best practice-type build.

Is your Netscaler physical or virtual?  If it’s a VPX, you need to upgrade to at least 10.5 build 57.7.  11 would be better, but read the release notes first.  Yes, NS11 code is out.  It’s been out for a little over a month now.  I have several of them running, both MPX and VPX.  They are stable, it works great, go get it.  Anyhow – The steps:

Disable SSLv3 and TLSv1 on any vServers:  This one is pretty easy.  Just go to the SSL Parameters on the vServer and make sure TLSv11 and TLSv12 are checked, uncheck SSLv3 and TLSv1.  Be sure to test this with all the devices and different browsers that will be used to access the system.  Some older devices and browsers get bent when you disable TLSv1.  But, the business will have to choose what’s more important – coddling users still on iPhone 4s and Windows XP, or security.  That decision is usually above our pay grade anyway, thankfully.  We can pawn off the reason for the broken devices on the people that made us do it, right?

Removal of eTag headers…  This one proved to be a bit more challenging.  WTF is an eTag header?  Why are they there?  How do I get rid of them?  I’m a virtualization engineer, and I work on Netscaler a bit.  Sure, I might know it better than some people, but I’m not a network engineer.  I’m not a web developer.  I don’t have an intimate knowledge of the HTTP protocol and GETs and POSTs and all this nonsense, and about the closest I come to dealing with headers is X-Forward-For.  So, when I ran across this one, dealing with it was a new one for me.

Go to web-sniffer.net.  Put in a URL, and it’ll come back and give you all the header info.  If you see eTag down there, chances are that the PCI scan company will be angry about it, since it can apparently be used to discover PID info.  Great.

To get rid of this, I used a Rewrite policy.  Pretty easy, actually.  Make sure Rewrite is enabled (hint: it’s under AppExpert).  Then create a Rewrite action and name it “eTag_Action”.  Type is DELETE_HTTP_HEADER, and the Header Name is, you guessed it:  ETag.

Now, create a Rewrite Policy.  Name it something intelligent like eTag_Policy.  The Action should be the action you just created, eTag_Action.  No log action, and leave the undefined-result action alone.  Under expression, enter

HTTP.RES.FULL_HEADER.CONTAINS(“ETag”)

Then, just go back to your vServer and link this policy.  It’s a REWRITE RESPONSE policy, BTW.

So, for simplicity’s sake, we’ll assume you’re only dealing with an AG vServer.  You link the policy there, then you point web-sniffer at https://host.domain.com/vpn/index.html and there are no eTag headers.  Hooray!  You’re done, right?

Not quite.  Scanning companies are pretty smart, and they’ll scan the individual .js pages (enter https://host.domain.com/vpn/js/gateway_login_form_view.js into web-sniffer), and look at the 200 responses even.  Crazy, but hackers would do the same thing, so they are just doing what you pay them to do, PITA or not.

So why are there still eTag headers being returned on the .js pages if you told the vServer to strip them?  Because of the caching policies on the AG vServer.  Remove as many as the GUI will let you (there will always be 2 that can’t be removed).  The caching policies are what injects the eTag response even AFTER you stripped it.

Now, if this is a vServer that requires caching, well, you’re going to have to take that up with your internal security guy, the PCI vendor, and whoever has something hiding behind that vServer.  I’m just a lowly little Citrix Geek, not a security guy.  I can’t fix all the world’s problems, but I’ll share the fixes I have found.

Compliantly,
CG1

PVS 7.6 Cache Disks on 2012R2

Posted by citrixgeek1 on

So…  Stood up some 2012R2 XA7.6 servers the other day, using PVS7.6 and cache in RAM with overflow to disk.

Then I saw that I had files in the WriteCache directory on the PVS server.  WTF?

After some screwing around, I found the solution.

Apparently PVS can’t use a disk that is GPT, you must use MBR.  Once I switched that and reformatted the disks, viola!  Everything worked exactly as it should.

Happy provisioning,

CG1

Receiver 4.2 desktop shortcuts

Posted by citrixgeek1 on

Yeah yeah, long time no blog.  I know.  Life gets in the way.  Work gets in the way.  I have a list of things I need to blog about, and I’ll get to a few of them here shortly.  For now, though, this one really needs to get out there, so I’m shortcutting said list, and getting it out there for you.  You’re welcome.  🙂

There seems to be very little information out there on this topic.  I Googled for hours.  I read the Citrix eDocs.  I read everything I could find.  Sure, lots of other people have blogged about it, but all they did was parrot Citrix’s announcement of the fact that the feature is back.  I highly doubt that even one of them has tried to implement the feature.  Or at least, if they did, they found out how big of a pain in the ass it is, and how the behavior of it is so strange that maybe they are best served by not telling the public about it.

As an anonymous blogger, I don’t have those same worries – making Citrix mad at me.

So, here I am to give you the “straight dope”, as it were.

 

First – the ADM template that is referenced to be used – I tried it.  I verified that it was applying properly by using GPRESULT /H.  It was working fine to apply the SSO settings.  GPRESULT showed all the shortcut settings I had configured.  Except there were no shortcuts.  It wasn’t doing what it was supposed to do.  The registry entries below were not configured.  NO idea what the deal is with that, but in the environment I was working in, it didn’t get the job done.  Now, maybe there was an environmental issue causing it?  I don’t know.  What I DO know is that I had to find a way to make it work, and so I’ll share that here with you just in case the ADM template doesn’t work for you, either.

HKLM\Software\Wow6432Node\Citrix\Dazzle

PutShortcutsInStartMenu  REG_SZ  true
PutShortcutsOnDesktop    REG_SZ  true
StartMenuDir  REG_SZ  <whatever the directory you want apps to go into in the start menu – leave blank to put them directly in start>
DesktopDir  REG_SZ  <whatever the directory you want apps to go into on the desktop – leave blank to put them directly on the desktop>

 

SO – What’s this weird behavior?  Well, for starters, it actually has to do with Storefront, not Receiver.  But since Storefront and Receiver function more or less as a pair, and the issue is manifest in Receiver, I’m still blaming that.  Mostly because I’m stubborn, as the issue DOES lie with Storefront.  For the record, this environment was using SF2.6.

The problem is that when Storefront remembers the application subscriptions, apparently a flag is set BY RECEIVER that tells Storefront whether or not it should create the desktop shortcuts.  Thus, if you have Receiver 4.0 or 4.1, and have a bunch of applications subscribed, THEN you upgrade to 4.2 – NOTHING HAPPENS.  No desktop shortcuts, no start menu folder like you specified – NOTHING.  How to fix this?  The official answer from Citrix (that doesn’t seem to be documented ANYWHERE that I can find) is “unsubscribe and resubscribe to all your applications”.  Well isn’t that just peachy.

Ok, fine.  So you’ve done that.

But then – you log into another workstation, and that one isn’t configured for desktop shortcuts.  Then you log back into one that is, and lo and behold – NO SHORTCUTS!

This is just ridiculous.  Why is the data about whether or not to publish an application shortcut stored in Storefront to begin with?  What are the implications for people who use XenDesktop, and access that environment using the HTML5 client?  What then?  I didn’t test this, but I’d imagine it will create a problem as well since naturally the HTML5 client relies on Storefront subscriptions, and is incapable of publishing a shortcut to the desktop on the client machine the browser is being run on.

IMO, the way this SHOULD work is Storefront just keeps track of the subscribed apps, and the Receiver handles creating and publishing shortcuts for any subscribed app.  That way moving amongst machines with different Receiver versions and/or configurations won’t create a problem.  I mean really, Citrix.  You bill yourself as a mobility company, and yet when the rubber meets the road, now you’re saying that “Sure, you can be mobile – as long as all your access devices have the exact same version of Receiver configured the exact same way.”

 

Doesn’t sound very mobile to me.

 

Good luck,

CG1

Couple of Netscaler command-line tips

Posted by citrixgeek1 on

Spent some time on the phone with Netscaler support over the past few weeks, and made a note of two commands they used that I found useful.

The first will tell you, in real-time, what policies are hitting when a user logs in via Netscaler:

nsconmsg -g pol_hits -d current

The next will show you traffic, in real time, and let you filter it on IP:

nstcpdump.sh host <IP Address>

Not sure if anyone else will find these useful, but I’m sure I’ll be referring back to this post to use them in the future.

Load Balanced,
CG1CG1

Comments Off on Couple of Netscaler command-line tips Posted in NetScaler Tagged

PVS 7.6 issues with v10 VMs on vSphere 5.5

Posted by citrixgeek1 on

You know, there are only so many delays I’m willing to deal with in a day.

First, there’s the bug earlier that bit me during install.  Can’t have a space in the name of the OU.

Now, I find another one that gave me the redass.  HARD.

So you’ve got vSphere 5.5.  Excellent.  Citrix says it’s supported.  Everything looks fine.  The customer wants v10 VMs, which is a pain (mostly because VMware’s web interface is a kludgy, bug-ridden POS), but whatever.  NOTE:  Yes, I’m a VCP, too, so don’t think I’m just “hatin on the competition”.  It does need work!

So you build your base image, optimize it, and install the PVS Target device driver.

Reboot, and it hangs loading windows.  I actually removed the bootux disabled entry using bcdedit just so I could see what was going on.

What’s the problem?

With v10 VMs, VMware attaches the virtual CDROM using SATA, not IDE.  Apparently the PVS target device driver can’t deal with that, so the VM never finishes loading.  NOTE:  It ONLY does this when there’s a vDisk attached – if you remove the vDisk from the target device, Windows will boot every time, so it’s not like the driver just outright breaks something.  Even more infuriating.

The solution?  Switch the CDROM to IDE.  Then, don’t forget to remove the SATA adapter from the VM.  Then after you’ve done that, make sure you go into device mangler and remove all the dead stuff – the SATA adapter itself, as well as any ATA channels that are no longer present.  You should still see two ATA channels present after the removal.  Basically, you want to remove all the grayed out items.  How?

Open an administrative command prompt, and enter “set devmgr_show_nonpresent_devices=1”.

Then, “start devmgmt.msc”

Then click view, and then show hidden devices.  Then expand IDE/ATA adapters, and remove all that stuff.

devmgr

Again, remove only the grayed out items.

While you’re in there, check the Network Adapters, and remove all the grayed out NICs, too (but you already did that, right)?  *IF* you found any grayed out NICs and removed them, you should uninstall and reinstall the target device driver to ensure it binds to the correct NIC.

Then go ahead and re-run the imaging wizard, and you should FINALLY be able to pull an image of your VM.

Me?  I’m pretty disappointed in Citrix.  vSphere 5.5 has been out for a while now, and PVS 7.6 was only just released a couple months ago.  One would think they could have accounted for this, or at least made prominent note of it somewhere telling people about the problem.

But alas, here I am having to blog and complain about it.  Maybe next time..

Provisioningly,
CG1

PVS 7.6 bug

Posted by citrixgeek1 on

So, I’m doing my first production build of PVS 7.6 servers for a customer.  This particular customer had an OU already defined for all of their security groups. Ok, no problem, I’ll put the security groups in there for farm administrators and such.

Yeah, no.

The OU had a space in it.  The installation took just fine, but then it would not let me into the farm.  I got the old “This domain/user does not have access to the farm”.  Gee, thanks.  So, I go check the dbo.AuthGroup table.  It had a single entry, and it was correct:  “Domain.com/security groups/group”.

I moved the group to an OU without a space in the name, deleted the database, and re-ran the config wizard to create a new farm, and whuddya know?  It all started working again.

Even though Citrix wasted an hour of my time with this, hopefully you won’t waste yours.

Provisioningly,
CG1

PVS and DST

Posted by citrixgeek1 on

Apparently PVS has some issues with DST.  Time doesn’t get updated correctly on targets, duh.  How does this manifest?  Event logs out of order, GPOs not getting applied, etc etc.

 

How do we address this?  Well, in PVS-land, one of the most important GPOs you have is the one that says “Do not change machine account password”.  If that one doesn’t work, no one can log into any of the targets because domain trust is messed up, right?  Well…   Imagine if for some reason your GPOs failed to apply!  What then??

 

Full info here:  http://support.citrix.com/article/ctx123336

 

  • Change the image mode to private image and check that the Local Security Policy Domain member: Disable machine account password changes is enabled within the vDisk image. — this is important so that the machine account password policy remains in effect even if GPOs aren’t correctly applied.
  • Resynchronize the computer before you restart.

Then to fix the time problem:

  • Click Start > Run. Enter cmd to open the command window.
  • Run the following commands:
w32tm /config /update
w32tm /resync
Then restart, log in, and I recommend a gpupdate /force.  Then, shut down, flip back to standard, deal with KMS as necessary (see previous post), and you should be good.
-CG1