Wednesday, October 12, 2011

How many public-facing authentication systems does the Australian government have?

From CNN Money, “The Welshman, the Walkman, and the salarymen” 1/06/2006

“Rob Wiesenthal, a top deputy to Stringer [CEO of Sony] who is in charge of [Sony worldwide] strategy and M&A, says, "I have 35 Sony devices at home. I have 35 battery chargers. That's all you need to know."


I feel the same way about the Australian government and authentication services which got me wondering, how many usernames and passwords could an Australian citizen have with the government? With the advent of OpenID, Shibboleth and a consumer acceptance of SSO offerings like Facebook Connect and Messenger Connect, how many Australian government agencies are still entrenched in the local authentication mindset?

For the purposes of this exercise, I will limit the scope to
  • Only federal government agencies – no state or local agencies, no wholly owned private companies

  • Only services where a citizen or company can register – no internal only resources that are publically accessible (ie. no extranets), no G2B or G2G services. If there is a publically accessible registration process, it counts.

  • The method of registration doesn’t have to be online – only needs to be available to a citizen or company.

  • Only services which require a username and password – no e-mail subscription lists, surveys.

  • No universities – I want to publish this blog post this century

  • Anything that meets the criteria above counts: doesn’t matter if it’s the ATO web portal or a single Jira instance used by two people.

I started by listing the online services I was aware of. Then I used the Google query "forgotten password" site:gov.au -site:vic.gov.au -site:nsw.gov.au -site:wa.gov.au -site:sa.gov.au -site:act.gov.au -site:qld.gov.au –site:tas.gov.au –site:nt.gov.au


Site Login Register Forgotten password?
Australia.gov.au
AusKey
Medicare Australia
Medicare Australia – Health Professional Online Services
forgotten password?
Centrelink
Australian JobSearch
Child Support Agency
Australian Securities and Investments Commission
None, call service desk
National Health and Medical Research Council – Research Grants Management System
Login
National Archives of Australia RecordSearch
None – send e-mail to a reference officer
Australian Research Council – Research Management System
Department of Education, Employment and Workplace Relations – Endeavour Awards
Department of Finance and Deregulation – Govdex
Employment & Community Services Network
Resource training generator
Prime Minister’s Department – E-mail subscriptions
Department of Veteran’s Affairs – Secure Services
None
None
Federal Register of Legislative Instruments – Lodgement Portal
None
National Library of Australia
None, call help desk
Medicare Small Business Superannuation Clearing House
Senate – Senate Committee Submissions
Parliament of Australia – ParlInfo Search
Australian Public Service Commission – APS Jobs
Department of Sustainability, Environment, Water, Population and Communities – Australian Bird and Bat Banding Scheme
Department of Sustainability, Environment, Water, Population and Communities – Water Efficiency Labelling and Standards Scheme
Department of Sustainability, Environment, Water, Population and Communities – Australian National Shipwreck Database
Register
AusTender
Federal Court of Australia – eLodgment
National Transport Commission – Online Portal
Australian Education International – AEI Online
AusTrade
Australian Bureau of Statistics – TableBuilder
Register – need to fax a form
Australian Bureau of Statistics – CensusAtSchool
Fair Work Australia – eFiling
Australian Bureaeu of Statistics – MiCRO
Insolvency and Trustee Service Australia – Online Services
forgotten password?
Department of Innovation, Industry, Science and Research – The Prime Minister’s Prize for Science
Department of Innovation, Industry, Science and Research – International Science Linkages
Department of Education, Employment and Workplace Relations – School Services Point
Tax Practitioners Board
Australian Customs and Border Protection Services – Careers and Recruitment
Register – you get an account during the application process
Australian Council for the Arts – Online Services
Australian Government - eSub Online
forgotten password?
Australia War Memorial
Aged Care Australia – my page
Department of Education, Employment and Workplace Relations
Department of Defence – Defence Science and Technology Organisation – DSTO Publications Online
National Ethics Application Form
National Measurement Institute
Australian Customs and Border Protection Service – Subscriptions
Department of Innovation, Industry, Science and Research
Australian Nuclear Science and Technology Organisation – Publications Online
My Plan Indigenous Opportunities Policy
Council for the Australian Federation – Online Services
Department of Innovation, Industry, Science and Research
Department of Families, Housing, Community Services and Indigenous Affairs – FLoSse Research
Ageing Research Online
Austrade - Marine
Australian Electoral Commission – eReturns
Australian National Maritime Museum – Australian Register of Historic Vessels Forums
MyFuture
Museum of Australian Democracy at Old Parliament House
Department of Innovation, Industry, Science and Research – Australia-India Strategic Research Fund
Department of Climate Change and Energy Efficiency – Online System for Comprehensive Activity Reporting
National Library of Australia – Trove
AusIndustry
National Film and Sound Archive
Department of Immigration and Citizenship – Newsroom
Australian Fisheries Management Authority – Quotaboard
Australian Pictorial Thesaurus
forgotten password? – None, contact APT Coordinator
Civil Aviation Safety Authority – DAMP Reporting
forgotten password? – None, contact AOD team
Civil Aviation Safety Authority
Australian Institute of Health and Welfare – METeOR
Australian Prudential Regulation Authority – National Claims and Policies Database
National Health and Medical Research Council – Emergency Care Information Gateway
Australian Broadcasting Corporation – Communities
Department of Innovation, Industry, Science and Research – DIISR Authentication Gateway
Defence Housing Authority – Online Services
Register – or call DHA
Department of Health and Ageing – NICNAS
forgotten password?
National Archives of Australia – Vrroom
Business.gov.au – Business Consultation
Lending Rights
Energy Rating – Online Services
Australian Health Practitioner Regulation Agency – Online Services
Register – through online form
Development Assessment Forum – eDA
Office of the Renewable Energy Regulator – REC Registry
Market Based Instruments
Australian Sports Commission – Athlete Training System
Australian Communications and Media Authority – Events
Register – registration upon ticket purchase
Australian Sports Commission – ACEonline
forgotten password? – none, webform to administrator
Sugar Research and Development Corporation
Disability Policy & Research Working Group
Register - YOU CAN LOGIN WITH OPENID!!!
Living Greener
Register – YOU CAN LOGIN WITH OPENID!!!
Director of National Parks
National Museum of Australia
Department of the Environment and Water Resources
National Film & Sound Archive
forgotten password?
Australian Communications and Media Authority – Number Planning Inquiry Registration
forgotten password?
Australian Law Reform Commission
Register – YOU CAN LOGIN WITH OPENID!!!
Department of Education, Employment and Workplace Relations – myUniAssist

I decided to quit after 100. The answer? Probably a few hundred. What does this mean? On the upside, it’s nice to see that the federal government has a lot of online services. On the downside, it’s disappointing that they don’t have an SSO strategy, and doubly disappointing that the government doesn’t think that federated login is necessary (I’m guessing that statement was made to set the success level for australia.gov.au very low.)

In the meantime, I’m going to login to my Foursquare account with my Facebook credentials, and signin to my TripIt account with my Google credentials.

Tuesday, August 16, 2011

The basics of ordered linked lists in C

My C lecturer said my linked list implementation was the best linked list code he'd seen written under exam conditions in his career. True story. The question involved writing a linked list in C to manage books in a bookshelf (each book was an instance of a struct). But unlike my fellow students who took time to memorise linked list code, I didn't bother as I was too busy playing Civilization. Priorities people!

If you're sitting a C exam that has linked lists, you can expect a generic question about linked lists. The question is usually in the form "You have a bunch of (objects) in a (collection). Write a linked list to store this data." The objects are usually something like
  • books in a bookshelves
  • words in a sentence
  • phone numbers in a phone book
  • spells in a spellbook if your lecturer likes Harry Potter.
The logic behind each is the same, and the only thing that changes is the name of the variables and the amount of variables in the struct. If you understand one linked list, you'll understand them all.

Nodes

In algorithms and data structures terminology, the items in a linked list are called nodes. If you're having trouble conceptualising a node, think of it as a business card. Just like a business card, a node can contain a single piece of data on it (like "MIB") or it can contain multiple types of data (like a name, address, phone number).

A business card with a single piece of data
In addition to the useful data you might put in a node, each node also contains a pointer to the next node in the list. If you were a visual person, you might visualize this as a piece of string connecting two business cards.

Linking the nodes

All nodes in a linked list are connected to each sequentially using pointers. This is why they are called linked lists. The first node points to the second node. The second node points to the third node, and so on and so forth until it gets to...

The last node

If it's the last node in the list, the pointer points to NULL. This is important, because checking for NULL is the way we know we're at the end of the list.

Is there any way of skipping to the 27th node?

No. You go to the head node, and then you follow the pointer to the next node. Then you do that 26 more times.

The head node

You'll need a way to find the first node in the list. This is known as the head of a linked list. The head tells you where the linked list starts: if you lose the contents of the head node, the whole list is screwed and you're a buffoon. Don't be a buffoon! Tips for not being a buffoon:
  1. When creating the first node in a linked list, the first node is the head.
    Point the head at the first node.
  2. When deleting the first node in a list, point the head at the second node.
The current node

As you iterate through a linked list to look at its contents, you'll need a way of tracking which node you're looking at. This is the current node. To cycle to the next node, we change where the current node points with

Current = Current->Next

Tuesday, July 5, 2011

Configuring license sources for FlexLM, FLEXnet, Flexera, FLEXenabled, Flex apps

Applications that are "protected" with FlexLM/FLEXnet/Flexera licensing servers need network connectivity to the license server. But how does the application know where to look?

The FlexLM order of precedence

Applications look in several sources for the location of a license server. If more than one license server source exists, the following order of precedence is used:
  1. The vendor-specific FLEXnet environment variable
  2. The vendor-specific registry entry or configuration file
  3. The "global" LM_LICENSE_FILE environment variable
  4. Any .lic file in the license path
This means that if you've configured the global LM_LICENSE_FILE environment variable, the Flex runtime will ignore any .lic in the license path. When you are troubleshooting license check-out, make sure you're not unintentionally overriding your lower precedence license preferences.

Bonus points.

Lower precedence license preferences: say that 5 times fast for 10 internet points.

The "global" LM_LICENSE_FILE environment variables

The LM_LICENSE_FILE environment variable configures the license server information for all products that use the FLEXnet framework. The LM_LICENSE_FILE variable should be used sparingly because there may be more than one FLEXenabled app on the computer, and you don't want to accidentally override this.

The vendor-specific registry entry or configuration file

Most FLEXnet vendors have a vendor variable that can be used: for example, ESRI ArcGIS uses ARCGIS_LICENSE_FILE.

Configuring environment variables on Windows manually
To configure the LM_LICENSE_FILE environment variable manually, perform the following tasks.
  1. Go to Control Panel > System.
  2. In the System Properties window, go to the Advanced tab and click the Environment Variables button.
  3. Under the User variables and System variables, ensure there is no variable you are trying to create does not already exist. If it does, delete it.
  4. In the System variable section, click New.

  5. Enter the relevant variable name and value.

  6. Clients do not need to be restarted for changes to take effect. These setting become effective when the System Properties dialog box is closed. Yes, I've tested this.
Configuring environment variables on Windows via Group Policy

Environment variables can be configured via the registry. The registry entry is HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment. 
Registry keys or scripts that change system variables can be deployed via Group Policy (via custom ADM templates) or a package deployment system such as Microsoft SCCM/SMS or Altiris. 

Configuring system variables in Unix/Linux/MacOS
License variable can be set in Linux or MacOS by using the setenv or export command depending on the shell. You can either type the command in a shell session (which will make it only last for the session) or add it to the appropriate file in your home directory. Examples include:

C:
setenv LM_LICENSE_FILE=27000@licenseServer1

bash:
export LM_LICENSE_FILE=27000@licenseServer1

What should the environment variable be if a FlexLM triad is being used?

Specify the 3 servers separated by commas.

27000@licenseServer1,27000@licenseServer2,27000@licenseServer3

Should I use an FQDN in the environment variable?

Some versions of the FlexLM runtime don't understand FQDNs and just error out. MATLAB is one of them. This needs to be tested on an app-by-app basis.

Monday, July 4, 2011

Decommissioning a VMware ESX host with a Nexus 1000V VEM – why and how?

A decommission process for ESX hosts is essential for environments of any size. Without a decommission process, you risk leaving behind all sort of garbage: DNS records, CMDB entries, allocated IP addresses, etc. If you use Cisco Nexus 1000V virtual distributed switches (vDS) in your environment, you’ll need to have a few extra tasks in your host decommission process.

I don’t have a decommission process for my physical switch ports, why should I need one for my virtual switches?

The Cisco Nexus 1000V has several limits including maximum VEMs registered and maximum vEthernet interfaces. If you don’t properly decommission your hosts, you will be prevented from installing the Nexus 1000V VEM on additional hosts!

Cisco Nexus 1000V

A Cisco Nexus 1000V. If you zoom in close enough,
you can see all 8192 ports.

Surely if I remove/disconnect a host from vCenter, the ESX host’s VEM is unregistered from the VSM and the vDS ports are automatically freed up?

Nope! You will need to perform these three tasks manually

  1. Unlink the vDS uplink ports
  2. Remove your ESX host from the vDS
  3. Uninstall the VEM

Unlinking the vDS uplink ports

During the server decommission, you’ll need to remove the Nexus VEM from the ESX host. This should be performed after the host has been placed in maintenance mode but before it has been turned off. The first step is to disconnect the ESX host’s virtual uplink ports to the vDS.

  1. In the Networking screen (Home > Inventory > Networking), right-click on the vDS and click Manage Hosts

    VMware Networking - Manage Hosts
  2. In the Select hosts screen, select the ESX host you want to unregister then click Next.

    VMware Networking - Hosts connected to vDS - Select hosts
  3. In the Select physical adapters screen, unselect the physical adapters (vmnics) attached to your Nexus 1000V. In this case, vmnic2 and vmnic3 are used by the switch N1K.

    Once you have unselected the physical adapters click Next.

    VMware Networking - Hosts connected to vDS - Select physical adapters
  4. In the Network connectivity screen, click Next.

    VMware Networking - Hosts connected to vDS - Network connectivity
  5. In the Virtual machine networking screen, click Next.

    VMware Networking - Hosts connected to vDS - Virtual machine networking

  6. In the Ready to complete screen, click Finish to unlink the vDS uplink ports.

    VMware Networking - Hosts connected to vDS - Ready to complete
  7. If you receive a warning message, click Yes to continue. Don’t worry, you’ve already placed the host in maintenance mode so there’s no risk of VMs losing network connectivity.

    Warning: there are no physical adapters selected for one or more hosts. Virtual machines running on these hsots will encounter networking problems when trying to connect to this vNetwork Distributed Switch. Do you want to continue?

Now we have to remove the host from the vDS.

  1. In the Networking screen (Home > Inventory > Networking), select the vDS that the host was connected to.
  2. Click the Hosts tab.

    VMware Networking tab bar
  3. Right-click on the host then click Remove from vNetwork Distributed Switch…

    Right-click on a VMware host and select Remove from vDS
  4. Click Yes to continue.
    Because you’ve already placed the host in maintenance mode and removed all the uplinks, it is safe to proceed.

    Removing selected hosts from the vNetwork Distributed Switch might result in loss of network connectivity. Do you want to remove the selected hosts?

One last step! We have to remove the VEM from the host. This step unregisters the VEM from the VSM.

  1. Execute the following command at the ESX console.

    vem-remove –d

    Removing the Cisco Nexus 1000V from the service console command line: vem-remove -d

All done! You can continue with your usual ESX host decommission process.

Now, if you haven’t followed these steps correctly, you might receive the error message

Not removing VIB: Nexus 1000v switch found (N1K)
Please remove host from DV switch or run hotswap script.

vem-remove -d error: Not removing VIB: Nexus 1000v switch found (N1K) Please remove host from DV switch or run hotswap script.

If you get this, you haven’t performed steps 8 to 11. Remember, you have to remove the host from the vDS before you unregister the VEM.

Tuesday, June 14, 2011

iiNet and AFACT

From ZDNet, 2 June 2011: Content owners don't back AFACT

If true, it's a welcome change. The fact that copyright infringement of “Hollywood” (AFACT) films occurs should not necessitate or compel imposing an obligation upon ISPs to become unpaid “Hollywood” copyright police, merely because they feel something must be done to stop the infringements.  As AFACT have chosen to redefine civil copyright breach as “theft” (their chosen name!), let them fund their own civil copyright theft police. They cannot expect the law (which until now has recognised no positive obligation on any person to protect the copyright of another) to impose commercially burdensome obligations upon an industry that is lawfully providing a communication service.  

Disclosure: I have been a happy iiNet customer for 8 years.

Monday, June 13, 2011

ISO 8601 date/time/duration manipulation with XSL

I have an XML log full of events (yay). The vendor have chosen to represent events and event durations with two XML variables: eventStart and eventDuration. My challenge: I need to transform the following XML


<eventStart>2011-07-13T18:00:00</eventStart>
<eventDuration>PT1H30M</eventDuration>
<eventDescription>Cisco Burger Maker cannot make burgers</eventDescription>


into CSV that can be scraped by another application


7/13/2011,18:00,19:30,"Cisco Burger Maker cannot make burgers"


You might comment "that eventDescription looks normal, but what kind of silly notation are eventStart and eventDuration in?!" It's ISO 8601 which is the standard for interchange of date and times. It's commonly used in XML documents prevent a Abbott and Costello "Who's on first?" ambiguity when representing date, time and duration.


"One second? No, I need you to tell me the duration now!"

The XML contains the eventStart and eventDuration, but no end time. To produce the output I need, I'm going to need to do some date manipulation. This would be easy enough in any other languages: Java and C# have classes that deal with date manipulation. Unfortunately, XSL isn't as flexible. To do the sort of transformations requires to get the output, we'll need to do a bit of string hacking. To ease the string happening, I've expended all my photoshop skills to produce this diagram that shows the character positions.

I installed Photoshop for this?!

Let's get manipulating!


1) How do you convert an ISO 8601 date to DD/MM/YYYY?
We can do this with simple string manipulation. Because ISO 8601 requires padding of date variables (ie. the Queen's birthday is stored as 2011-06-13 and not 2011-6-13), we are guaranteed that that the first four characters are the year, the 6th and 7th are the month, and the 9th and 10th are the days. You can use the substring command to grab the right characters, some / characters to separate them, and the concat command to glue it all together.


<xsl:value-of select="concat(substring(.,9,2)),'/',substring(.,6,2),'/',substring(.,1,4))"/>


Using that operation could result in the output 02/05/2011. What if we want to drop the preceding zero (ie. get 2/5/2011)? The number function does that.

<xsl:value-of select="concat(number(substring(.,9,2))),'/',number(substring(.,6,2)),'/',number(substring(.,1,4)))"/>


2) How do you get the time from an ISO 8601 date?
This can be performed with easy string manipulation. We can use substring to grab all the five characters after the T and stick them into a new variable called start-time.

<xsl:varaible name="start-time" select="substring(substring-after(.,'T'),1,5)"/>


Applying this to 2011-06-13T18:30:00 gives 18:30.

3) How do I convert an ISO 8601 duration into a 24hr duration?
For the purposes of simplicity, I'm going to assume that your periods contain only hours ('H') and minutes ('M') (ie. your period will either be in the form PT30M, PT1H, PT1H30M). No days/weeks/months/years.

To do this, I'll create three variables.
  • duration-dirty will contain the duration in ISO 8601 format, except with the PT and M characters removed. I'm using this variable to reduce the amount of substring and translate functions in the later code.
  • duration-hour will contain the hour digits
  • duration-minute will contain the minute digits
Here's a diagram that shows these variables relation to the original eventDuration element.

To get dirtyDuration, we can use the substring functions to perform some slicing and dicing. To start, we can get rid of the PT and M characters.

<xsl:variable name="duration-dirty" select="translate(translate(eventDuration/text(),'PT',''),'M','')"/> 

Once we've done that, the period will look something like 30 (30 minutes), 1H (1 hour) or 1H30 (1 hour and 30 minutes. We can determine whether the duration contained hours by converting the duration-dirty variable to a number. If the conversion outputs NaN (not a number), we know there were more than 60 minutes in the duration.

<xsl:variable name="duration-hour">
     <xsl:choose>
          <!-- If it's not NaN (ie. a valid number), then the hours are 0 -->
          <xsl:when test="not(string(number($duration-dirty)) = 'NaN')">
               <xsl:text>0</xsl:text>
               <xsl:otherwise>
                    <!-- If it's NaN, there were hours. Use substring-before to grab anything before the H. -->
                    <xsl:value-of select="substring-before($duration-dirty,'H')"/>
               </xsl:otherwise>
          </xsl:when>
     </xsl:choose>
</xsl:variable>

Calculating the minutes is same same but different: we check if the duration-dirty element can be converted to a number. If it can, then dirty-duration contained only minutes (so we can use it). If converting it to a number returns an NaN, there were hours so we need to grab everything after the H.

<xsl:variable name="duration-minute">
     <xsl:choose>
          <!-- If it's NaN, there are no hours. duration-dirty is good to use. -->
          <xsl:when test="not(string(number($duration-dirty)) = 'NaN')">
               <xsl:value-of select="$duration-dirty"/>
          </xsl:when>
          <xsl:otherwise>
               <!-- If it's not NaN, then there are hours! Grab everything after the H. -->
               <xsl:value-of select="substring-after($duration-dirty,'H')"/>
          </xsl:otherwise>
     </xsl:choose>
</xsl:variable>

4) How do I add times together?
Suppose we want to calculate the end time of an event given eventStart and eventDuration. Step 2 will give us 18:00 from 2011-07-13T18:00:00. Step 3 will give us the variables duration-hour and duration-minute (1 and 30 respectively). But how do we add these two?

Start by calculating the end hour. If we add duration-minute to the minute digits in start-time and exceed 60, an hour has passed. And if we have more than 24 hours...go back to zero using the modulo function! The modulo function is sorta like the the math equivalent of word wrap: 22 mod 24 = 22, 23 mod 24 = 23, 24 mod 24 = 0, 25 mod 24 = 1. Perfect for 'resetting' back to 0.

<xsl:choose>
     <xsl:when test="substring($start-time,3,2) + $duration-minute > 60">
          <!-- An hour has passed! Add an extra hour -->
          <xsl:value-of select="(number(substring($start-time,1,2)) + $duration-hour + 1) mod 24"/> 
     </xsl:when>
     <xsl:otherwise>
          <!-- An hour has not passed. Just add the hours together. -->
          <xsl:value-of select="(substring($start-time,1,2) + $duration-hour) mod 24"/> 
     </xsl:otherwise>
</xsl:choose>


Awesome! But...if you have less than 10 hours, your output won't look pretty (ie. we want 09:30 rather than 9:30). We can easily fix this by padding a zero character if the hours are less than 10.


<xsl:if test="((number(substring($start-time,1,2)) + $duration-hour + 1) mod 24) &lt; 10">
     <xsl:text>0</xsl:text>
</xsi:if>


Good. Now calculate the end minute. I'm no physics major but if I recall correctly, there are only 60 minutes in an hour. If there are 60 minutes, the hour increments and the minutes reset back to 0.

<xsl:choose>
     <xsl:when test="not(string(number($duration-minute)) = 'NaN')">
          <!-- More than 60 minutes - go back to 0! -->
     <xsl:value-of select="number(substring($start-time,3,2) + $duration-minute) mod 60"/>
     </xsl:when>
     <xsl:otherwise>
          <!-- Less than 60 minutes. Easy. -->
          <xsl:value-of select="substring($start-time,3,3)"/>
     </xsl:otherwise>
</xsl:choose>

Awesome! This code assumes that your event starts and ends during the same day. I'll leave incrementing the day as an exercise for you. Not because I don't know how, but because my '<' key is playing up!