Category: IT Stuff


My company provide SQL dba services and build system integrations. We use various VPN clients to connect in to most company networks but have always had issues using integrated windows (active directory) authentication with certain applications.

Launching an RDP session or browseing a network share works just fine, but if we want to connect SQL Server Management Studio to a server on the remote network using our domain credentials we have previously been stuck. Leading us to deviate from (what I feel is) best practice and create sql users.

When we build interfaces in Visual Studio they will normally run on client servers using active directory/domain service accounts. If we wanted to carry out any troubleshooting we could not accurately emulate the interface in Visual Studio (as the domain service account).

If our local machines were joined to the client domain we could either log on as the service account or run SSMS / VS as the network user, but otherwise we are out of luck…

…that was until we stumbled across the /netonly command line argument for runas.exe.

Apparently this whole time there has been a technique to launch an application using credentials which don’t exist locally or on the current/trusted domain! We now have some .bat files saved which launch our core applications using runas.exe with the /netonly argument. For example;

runas /netonly /user:remotedomain\remoteuser “C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\ManagementStudio\ssms.exe”

I recently configured my Ubiquiti/Ubnt Unifi wireless access point to use WPA Enterprise (wpaeap) and pointed Radius at my domain controller running Network Policy Server (NPS).

I could connect fine using my Android mobile but could not connect from my laptop. Logs in the event viewer indicated an authentication issue, but this was definitely not the case.

After lots of fiddling and googling I discovered that PEAP does not work with wildcard SSL certificates. I replaced the certificate with a server specific cert and voila.

Here’s an article which shows you where to change the certificate; https://cantechit.com/2015/07/10/windows-nap-as-radius-in-a-windows-7-server-2012-wireless-world/

I have recently switched my network configuration from 2 routed subnets;

192.168.0.0/24
192.168.5.0/24

To a single subnet;

192.168.0.0/21

I had hoped this would have little to no impact and be a seamless transition (how wrong I was). I have a mix of devices with both dynamic (DHCP) and static IP addresses. Those using DHCP didn’t cause much of an issues, but those configured using static addresses required the subnet mask and default gateway changing (again, fairly straight forward).

The real issue came when it came to changing my vCenter Server Appliance (VCSA) network configuration. The obvious place to look (and several articles online) pointed toward the configuration option inside the vSphere Web Client; System Configuration -> Nodes -> x ->Manage -> Settings -> Common -> Networking

vc_network_settings

Unfortunately the settings are grayed out with a message “IPv4 configuration for nic0 of this node cannot be edited post deployment.”.

Other articles pointed toward the console (alt-f2 option), vmware Appliance Management Interface (VAMI) running on https://your-vc-hostname-or-ip:5480/ or SSH; Unfortunately I couldn’t login to try any of these techniques.

Console -> Alt-F2- “Authentication failed; Invalid login or password.”
SSH- “Login incorrect”
VAMI- “Unable to authenticate user. Please try again”

This had me stumped for many hours. I was able to reset the root password (reset the VM, prevent vcsa autoboot by pressing any key when the grub bootloader appears, press p, enter the grub password (default is vmware), enter, press e, add init=/bin/bash, enter, press b then type passwd root) but still couldn’t login using the new password. I think a few issues were at play here, but eventually tracked it down to the password complexity requirement forcing the use of special characters which were in turn being transposed by RDP / vSphere (” was becoming @ and £ was becoming # etc). Once I had figured the password issue I was then able to try the techniques again;

Console -> Alt-F2 -> Configure Management Network -> IP Configurationvc_consoleNope- “Configure Management Network; Management network configuration not allowed”

Next… SSH -> /opt/vmware/share/vami/vami_config_netvc_libxml2modNope- Lots of errors stating “ImportError: No module named libxml2mod”

Finally… VAMIvmaiAgain, no joy- “Updating has been disabled”.

Eventually I decided to try the same technique i’d previously used to modify my Linux (CentOS) VMs.

I started an SSH session and modified /etc/sysconfig/network/routes (from default 192.168.0.1 to 192.168.5.1 and then /etc/sysconfig/network/ifcfg-eth0 (NETMASK=’255.255.255.0′ becomes NETMASK=’255.255.248.0′). Rebooted and voila!

After moving from my home office to a real office I decided to downgrade my premium 80/20 business fttc connection (from claranet) to a residential 40/10 service from sky.

Yesterday, the connection was changed over and I found myself with no internet. I initially thought it was because the pppoe username and password needed updating (tail /var/log/messages was showing a CHAP authentication error message) but I don’t recall ever being sent a username/password. It was then I had a flashback to many years ago having to extract the details from a router/modem in order to use them in another device. A bit of googling backed this up, but also suggested the connection doesn’t use pppoe but MPoA and was going to be even more challenging to setup; http://www.skyuser.co.uk/forum/sky-broadband-fibre-help/51550-ubiquiti-edgerouter-lite.html

But this article was written in 2013, surely someone has documented the process more recently? Fortunately, before starting the long-winded process I stumbled across another aritcle; https://community.ubnt.com/t5/EdgeMAX/Sky-Fibre-DHCP-client-option-61/td-p/1172347. However, this seemed to point to needing a different modem (such as the Draytek Vigor 13) to achieve the MPoA connection.

Before I went and bought a new modem, I thought i’d try the BT Openreach/Huawei Echolife HG12. I deleted the pppoe interface from the Edgerouter and set the address on eth0 (connected to the modem) to DHCP. Still nothing… welll both of the previous articles did state the need to add the DHCP option; send dhcp-client-identifier "user|pass"; so I guess it’s time to unbox the Sky router and do some packet sniffing?

I must be in luck… 2 weeks ago, a post suggested you no longer need to use logon credentials, passing anything in the dhcp-client-identifier will do the trick. The example given was;

 client-option "send dhcp-client-identifier "bacons";"

So I gave it a try, but still no dice. Worth a reboot I guess? Power cycled the modem and voila, we have internet! Well, that was simpler than anticipated.

edgerouter_sky_fttc

Further to several posts about Raspberry Pi Digital Signage driving our office screens (https://tickett.wordpress.com/2014/12/03/anonymous-authentication-sql-server-reporting-services/), we recently restructured SQL Server Reporting Services. We updated the html/javascript file the Pi looks at (which redirects to SSRS) but it continued to try and use the old path.

This was fixed by SSH to the Pi then delete the browser cache;

pi@raspberrypi ~ $ cd /home/pi/.cache/chromium/Default/Cache/
pi@raspberrypi ~/.cache/chromium/Default/Cache $ rm -R *
pi@raspberrypi ~/.cache/chromium/Default/Cache $ sudo reboot

I flashed 5 of these phones back 6 months ago with some difficulty (after trying different methods/firmware versions/network cables/switches etc I finally got them all working), unfortunately I failed to record my findings! Time to resurrect the blog!

Last week I found myself with a few more Cisco CP-7906G IP phones to convert from old SCCP firmware to SIP. Rather than using the xml.conf method I chose to hard reset (I could remember this is how I did it last time);

  • Hold the # key while powering on the phone (either by POE or mains)
  • Once the red indicator on the handset start flashing release #
  • Key 3491672850*#

As my FreePBX (Asterisk PBX) is running and has the firmware ready I hoped everything would just work… Unfortunately not. All of the phones got stuck trying to pull term06.default.loads. I spent countless hours trying to figure out the problem, including;

  • Trying different DHCP servers (tftpd32, tftpd64, pumpkin & solarwinds TFTP server)
  • Trying different switches, network patch leads and even a direct connection between my laptop and the phones
  • Trying different IP ranges and network configurations

But still the phones repeatedly searched for term06.default.loads;

Dec 30 10:15:32 localhost in.tftpd[24929]: RRQ from 192.168.0.224 filename term06.default.loads
Dec 30 10:15:33 localhost in.tftpd[24950]: RRQ from 192.168.0.224 filename term06.default.loads
Dec 30 10:18:00 localhost in.tftpd[25093]: RRQ from 192.168.0.224 filename term06.default.loads
Dec 30 10:18:00 localhost in.tftpd[25094]: RRQ from 192.168.0.224 filename term06.default.loads
Dec 30 10:19:33 localhost in.tftpd[25151]: RRQ from 192.168.0.224 filename term06.default.loads
Dec 30 10:19:34 localhost in.tftpd[25152]: RRQ from 192.168.0.224 filename term06.default.loads

Eventually I decided to try a few different firmware versions “just in case”. Initially none worked (and they are hard to come by because I don’t have an active Cisco subscription and without one, you cannot download them directly from Cisco).

After almost giving up, I figured out that many of the firmware images I was previously unable to open (.cop or .cop.sgn) were in fact just zip files which can be opened in 7-zip. I recall when using the old technique to upgrade the firmware you need to start with a version around 8.5 then slowly incrementally patch. I didn’t think this was necessary with the “hard reset” technique, but found cmterm-7911_7906-sccp.8-5-2.cop.sgn and gave it a whirl….

Dec 30 10:44:41 localhost in.tftpd[26016]: RRQ from 192.168.0.245 filename term06.default.loads
Dec 30 10:44:42 localhost in.tftpd[26017]: RRQ from 192.168.0.245 filename jar11sccp.8-5-2TH1-9.sbn
Dec 30 10:44:49 localhost in.tftpd[26018]: RRQ from 192.168.0.245 filename cnu11.8-5-2TH1-9.sbn
Dec 30 10:44:51 localhost in.tftpd[26019]: RRQ from 192.168.0.245 filename apps11.8-5-2TH1-9.sbn
Dec 30 10:45:03 localhost in.tftpd[26024]: RRQ from 192.168.0.245 filename dsp11.8-5-2TH1-9.sbn
Dec 30 10:45:05 localhost in.tftpd[26025]: RRQ from 192.168.0.245 filename cvm11sccp.8-5-2TH1-9.sbn

Instantly the phone picked up term06.default.loads and proceeded to pickup the other files!

Once the phone booted, it clearly wasn’t going to register against my PBX (as it’s still running SCCP firmware). So I placed the SIP firmware cmterm-7911_7906-sip.9-4-2SR1-1.cop.sgn back on the TFTP. Carried out another hard reset and again, the phone instantly picked up the new files;

Dec 30 11:01:20 localhost in.tftpd[26726]: RRQ from 192.168.0.224 filename term06.default.loads
Dec 30 11:01:21 localhost in.tftpd[26727]: RRQ from 192.168.0.224 filename jar11sip.9-4-2ES9.sbn
Dec 30 11:01:28 localhost in.tftpd[26729]: RRQ from 192.168.0.224 filename cnu11.9-4-2ES9.sbn
Dec 30 11:01:31 localhost in.tftpd[26730]: RRQ from 192.168.0.224 filename apps11.9-4-2ES9.sbn
Dec 30 11:01:44 localhost in.tftpd[26736]: RRQ from 192.168.0.224 filename dsp11.9-4-2ES9.sbn
Dec 30 11:01:46 localhost in.tftpd[26737]: RRQ from 192.168.0.224 filename cvm11sip.9-4-2ES9.sbn

I provisioned the phones in FreePBX but they seemed to get stuck in a cycle registering/updating locale/rebooting. Fortunately I recalled having this issue previously and determined that the 7906 phones have a relatively short maximum password length which the default FreePBX passwords exceed. I was able to confirm this by looking at the Asterisk log;

[root@localhost ~]# tail /var/log/asterisk/full
[2015-12-30 02:06:30] NOTICE[1915] chan_sip.c: Registration from '<sip:1006@192.168.0.195>' failed for '192.168.0.198:51838' - Wrong password
[2015-12-30 02:08:07] NOTICE[1915] chan_sip.c: Registration from '<sip:1006@192.168.0.195>' failed for '192.168.0.198:51978' - Wrong password
[2015-12-30 02:08:54] NOTICE[1915] chan_sip.c: Registration from '<sip:1006@192.168.0.195>' failed for '192.168.0.198:49878' - Wrong password

Once I changed the password and rebuilt the configs in Endpoint manager the phones registered straight away.

In conclusion, I think the bootloader on the phones needed to be upgraded to enable them to be capable of loading the 9.x firmware. Flashing them with the 8.x firmware first upgraded the bootloader and from there it was plane sailing! I have attached the two firmwares for reference;

As a sidenote; I have never had luck with using DHCP option 150 (in either pfSense or tftpd32). Entering an IP address in it’s normal format has definitely never worked (when I do a wireshark trace I can confirm this). I believe you’re supposed to use some hex format, but having tried this in several different formats i’ve still never managed to get it working. The preferable route seems to be using DHCP option 66 (I believe this supports either a hostname or IP). In tftpd32 you don’t actually need to configure this option, the built in DHCP server automatically configures it to the hostname of the interface running the service.

Scrap my last post… Today I found I could no longer login to sharepoint and other federated services were acting strangely. I tried to troubleshoot but ultimately gave up.

Finally came the time to see if my weekly VM backups (and nightly incrementals) were actually any good… 15 minutes later and everything’s back to how it was 2 night’s ago!

Screen Shot 2015-04-21 at 15.48.06

Thankfully Veeam worked a treat and ADFS was back working perfectly BUT we still have our expiring SSL warning.

Screen Shot 2015-04-21 at 21.00.32

My theory as to what went wrong before… Whilst I had updated my SSL certificates on my on-premise ADFS server (including the token signing certificate), I think the Office365 hosted servers also need this same matching certificate. So, let’s try a different approach.

This time I followed a different link http://www.kraak.com/?p=190. I started the same way as before, replacing the SSL certificate in IIS. However, I didn’t update the token-signing or token-decrypting certificates in ADFS. Instead I issued the following commands;

Connect-MsolService
Get-MsolFederationProperty -DomainName tickett.net

At this stage, I can essentially see that both the ADFS Server (on-premise) and Microsoft Office 365 (hosted) certificates match;

Screen Shot 2015-04-21 at 21.16.46

At this point I noticed that, not only did they (obviously) bare the old dates, but they also bare the internal server name (and not the FQDN of either my old or new SSL certificates for IIS). This further backs up my theory that using the same certificate for IIS and ADFS yesterday was wrong. And in fact, it appears ADFS generates it’s own certificates when instructed to do so;

Update-ADFSCertificate -CertificateType: Token-Signing -Urgent:$true
Get-MsolFederationProperty -DomainName tickett.net

Screen Shot 2015-04-21 at 21.24.05

Right, we’re part way there. The ADFS Server certificate has now been renewed, but the Microsoft Office 365 certificate now needs to be updated;

Update-MsolFederatedDomain -DomainName tickett.net
Get-MsolFederationProperty -DomainName tickett.net

Bingo, now they both match! Exactly the same sequence of commands now needs to be executed for the token-decrypting certificate;

Update-ADFSCertificate -CertificateType: Token-Decrypting -Urgent:$true
Update-MsolFederatedDomain -DomainName tickett.net

And voila! I can now login perfectly to all my federated/SSO application/sites and Office365 webmail no longer warns me about a certificate approaching expiry.

*EDIT* This turned out to fail, please read the follow-up post; https://tickett.wordpress.com/2015/04/21/second-attempt-updating-adfs-ssl-certificate-on-windows-server-2012-r2/ 

I noticed a warning in Office 365 webmail that my SSL certificate was due to expire soon and hoped updating it would be a trivial task.

As always, I used https://www.startssl.com/ to generate a new certificate. I fired up IIS on the ADFS server and imported the new certificate (Server Certificates, Import). When I tried to bind the certificate to the ADFS https site I received a warning/error about a missing intermediate CA certificate. This was easily fixed by downloading the “Class 2 Intermediate Server CA” certificate from StartSSL and importing into the windows certificate store under Intermediate Certificate Authorities (Launched from the start menu by searching for “Manage computer certificates”);

Screen Shot 2015-04-20 at 17.07.17

Binding to the site in IIS was now successful. However, none of my federated applications were working. Just an ADFS error;

Screen Shot 2015-04-20 at 17.14.57

And some errors to match in the event log;

Screen Shot 2015-04-20 at 17.17.14

On each login attempted I was received the following 3 events;

Event: 111

The Federation Service encountered an error while processing the WS-Trust request.
Request type: http://schemas.microsoft.com/idfx/requesttype/issue

Additional Data
Exception details:
System.ArgumentNullException: Value cannot be null.
Parameter name: certificate
at Microsoft.IdentityModel.Threading.AsyncResult.End(IAsyncResult result)
at Microsoft.IdentityModel.Threading.TypedAsyncResult`1.End(IAsyncResult result)
at Microsoft.IdentityModel.SecurityTokenService.SecurityTokenService.EndIssue(IAsyncResult result)
at Microsoft.IdentityServer.Web.WSTrust.SecurityTokenServiceManager.Issue(RequestSecurityToken request, IList`1& identityClaimSet)

Event: 1000

An error occurred during processing of a token request. The data in this event may have the identity of the caller (application) that made this request. The data includes an Activity ID that you can cross-reference to error or warning events to help diagnose the problem that caused this error.

Additional Data

Caller:
TICKETT\lee

OnBehalfOf user:

ActAs user:

Target Relying Party:
http://adfs.tickett.net/adfs/services/trust

Device identity:

User action:
Use the Activity ID data in this message to search and correlate the data to events in the Event log using Event Viewer. This Activity ID will also be shown as additional information in the error page when an error occurs in the federation passive Web application.

Event: 364

Encountered error during federation passive request.

Additional Data

Protocol Name:
wsfed

Relying Party:
urn:federation:MicrosoftOnline

Exception details:
Microsoft.IdentityServer.RequestFailedException: MSIS7012: An error occurred while processing the request. Contact your administrator for details. ---> System.ArgumentNullException: Value cannot be null.
Parameter name: certificate
at Microsoft.IdentityModel.Threading.AsyncResult.End(IAsyncResult result)
at Microsoft.IdentityModel.Threading.TypedAsyncResult`1.End(IAsyncResult result)
at Microsoft.IdentityModel.SecurityTokenService.SecurityTokenService.EndIssue(IAsyncResult result)
at Microsoft.IdentityServer.Web.WSTrust.SecurityTokenServiceManager.Issue(RequestSecurityToken request, IList`1& identityClaimSet)
at Microsoft.IdentityServer.Web.Protocols.PassiveProtocolHandler.SubmitRequest(MSISRequestSecurityToken request, IList`1& identityClaimCollection)
at Microsoft.IdentityServer.Web.Protocols.PassiveProtocolHandler.RequestBearerToken(MSISRequestSecurityToken signInRequest, Uri& replyTo, IList`1& identityClaimCollection)
at Microsoft.IdentityServer.Web.Protocols.PassiveProtocolHandler.RequestSingleSingOnToken(ProtocolContext context, SecurityToken securityToken, SecurityToken deviceSecurityToken)
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.BuildSsoSecurityToken(WSFederationSignInContext context, SecurityToken securityToken, SecurityToken deviceSecurityToken, SecurityToken& ssoSecurityToken)
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.BuildSignInResponseCoreWithSecurityToken(WSFederationSignInContext context, SecurityToken securityToken, SecurityToken deviceSecurityToken)
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.BuildSignInResponse(WSFederationSignInContext federationPassiveContext, SecurityToken securityToken, SecurityToken deviceSecurityToken)
--- End of inner exception stack trace ---
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.BuildSignInResponse(WSFederationSignInContext federationPassiveContext, SecurityToken securityToken, SecurityToken deviceSecurityToken)
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.Process(ProtocolContext context)
at Microsoft.IdentityServer.Web.PassiveProtocolListener.ProcessProtocolRequest(ProtocolContext protocolContext, PassiveProtocolHandler protocolHandler)
at Microsoft.IdentityServer.Web.PassiveProtocolListener.OnGetContext(WrappedHttpListenerContext context)

System.ArgumentNullException: Value cannot be null.
Parameter name: certificate
at Microsoft.IdentityModel.Threading.AsyncResult.End(IAsyncResult result)
at Microsoft.IdentityModel.Threading.TypedAsyncResult`1.End(IAsyncResult result)
at Microsoft.IdentityModel.SecurityTokenService.SecurityTokenService.EndIssue(IAsyncResult result)
at Microsoft.IdentityServer.Web.WSTrust.SecurityTokenServiceManager.Issue(RequestSecurityToken request, IList`1& identityClaimSet)
at Microsoft.IdentityServer.Web.Protocols.PassiveProtocolHandler.SubmitRequest(MSISRequestSecurityToken request, IList`1& identityClaimCollection)
at Microsoft.IdentityServer.Web.Protocols.PassiveProtocolHandler.RequestBearerToken(MSISRequestSecurityToken signInRequest, Uri& replyTo, IList`1& identityClaimCollection)
at Microsoft.IdentityServer.Web.Protocols.PassiveProtocolHandler.RequestSingleSingOnToken(ProtocolContext context, SecurityToken securityToken, SecurityToken deviceSecurityToken)
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.BuildSsoSecurityToken(WSFederationSignInContext context, SecurityToken securityToken, SecurityToken deviceSecurityToken, SecurityToken& ssoSecurityToken)
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.BuildSignInResponseCoreWithSecurityToken(WSFederationSignInContext context, SecurityToken securityToken, SecurityToken deviceSecurityToken)
at Microsoft.IdentityServer.Web.Protocols.WSFederation.WSFederationProtocolHandler.BuildSignInResponse(WSFederationSignInContext federationPassiveContext, SecurityToken securityToken, SecurityToken deviceSecurityToken)

The earlier Office365 warning did link to a page which also included instructions for updating the certificates within ADFS, so I went ahead and did that too (In ADFS Manager, Set Service Communications Certificate, Add Token-Signing Certificate and Add Token-Decrypting Certificate). But still nothing… restarting didn’t help either.

Eventually I found an article with a few powershell commands; http://blogs.technet.com/b/tune_in_to_windows_intune/archive/2013/11/13/replace-certificates-on-adfs-3-0.aspx

Get-AdfsSslCertificate

Screen Shot 2015-04-20 at 17.24.02

Comparing this to the new certificate, I can see that it doesn’t match;

Screen Shot 2015-04-20 at 17.25.20

Set-AdfsSslCertificate -Thumbprint NEWSSLCERTIFICATETHUMBPRINT

Screen Shot 2015-04-20 at 17.26.18

Despite returning an error message referencing the old SSL certificate, you can see that the new one is now correctly assigned (by issuing the Get-AdfsSslCertificate command again).

And voila, I can now log in to my federated applications.

*EDIT* This turned out to fail, please read the follow-up post; https://tickett.wordpress.com/2015/04/21/second-attempt-updating-adfs-ssl-certificate-on-windows-server-2012-r2/ 

Kicking back for a week at Center Parcs. Came fully prepared with Raspberry Pi & 2TB Portable Hard Drive fully loaded but forgot the damn remote! My HTC One has an IR remote built in, but the Microsoft MCE remote isn’t listed. Every result googled turned up was an XBMC or Media Center remote which only works over the network/LAN.

Eventually, after trying a dozen or so IR apps with no luck, we found ZappIR.

zappir

After trying a few combinations- Console/Media Manager -> Microsoft -> Media Manager MCE Media Center Code Group 2- Bingo!

Now back to kicking back! Nice little hot tub, steam room and sauna in the back garden!

11042999_10155192464790567_1824434695954957278_n

%d bloggers like this: