Trustworthy Thoughts

The semester finished up for me a couple weeks ago. I taught the lab course this semester. There is no final in that course but instead a final project, so I’m done a bit earlier than regular lecture courses.

This was the fourth time through this course, and overall I thought the course went well. I ended up with 21 students, which was the highest enrollment I’ve seen. I don’t think I’d want to go much higher than 21 unless we had more resources. The students overall were quite strong, and I enjoyed interacting with them.

At the end of the semester I also got notified that the course now has a regular course number. I had been teaching it under a special topics number until now. I may never teach a cs498sh course again! Or at least not for a while.

• Virtual Machines - I used vmserver on a 64 bit FC8 base, and it was so much better than continually reloading images and dual booting. I got a set of new Dell systems, so this was the first year that running vmware was really feasible. I was able to build a image on one machine and then copy it around to the other machines. For some of the OS configuration labs, we were able to have each student have his/her own image. This made the SE Linux lab slightly better but not totally (see what went wrong below). The Department support guy at the beginning of the semester suggested just using vmware for everything. At that time, I didn’t really have any experience with vmware and was hesitant to completely give up on a physical lab. I think I’ll stay with the current environment for at least another semester. A good portion of the labs involve networking, and I think working with real devices is more beneficial that interacting with virtualized routers. I’m not familiar with how much variety vmware supports for virtual network devices. Plus having a physical lab forces students to directly interact with each other, which has fallen away from most of the other lab oriented courses.
• Metasploit and the exploit lab - Each semester I’ve had the students write a stack smashing attack. I’ve followed the framework of the Smashing the Stack for Fun and Profit article. I give the students a basic shell spawning shellcode, and give them the assembly for a more advanced shellcode that they must translate to hexcode. Last year, I had another professor guest lecture and give a demo of metasploit. This semester we had an in lecture exercise using metasploit at the beginning of the exploit lab assignment. A couple students figured out another way of using the metasploit generated poison packet to launch the attack. And in the shared lab environment, they taught this technique to their colleagues. Only three students used the technique I outlined in the lab writeup. I think this is cool because the students dug into metasploit. However, I think many of the students lost out on a deeper understanding of how the exploit really works because they didn’t have to translate to hex or create their own packet. I haven’t decided on whether I’m going to insist on a particular technique next time I give this lab.
• Vista Lab - Actually this was almost identical to the XP least privilege programming lab I’ve used in past years, but I added a component to look at the mandatory integrity controls (MIC) that Vista adds. The students also had to figure out how to work around the user account control. Based on the results of this semester’s labs, I should be able to integrate a real MIC part to the lab next semester.
• Virtual PIX - This is the first year I’ve used the virtual firewall (security context) feature of the PIX. Since I have only a very basic license, I could only set up 2-3 virtual firewalls per physical firewall (5 physical devices). But with that and VMs on the hosts, I could set up 10-15 reasonably separated firewall environments for pairs of students to work on. There was one major hicup that caused much student angst. For a each virtual firewall on a PIX, you want them to have unique MACs. You could assign your own, or use the “automatic” MAC. I went automatic and did not review the MAC selection closely. Alone in the lab, my few probes worked fine, but with a full lab traffic would fail mysteriously. Some students noticed that traffic destined for another students network was being delivered to their machine. Turns out the auto MAC selection only guarantees uniqueness within a device. Between devices it pretty much guarantees conflicts. Once I created my only MAC naming scheme, everything worked fine.

• SE Linux Lab - Each year I’m hopeful that the SE Linux lab will go well this year. In the first two years, we had file labeling problems because students were sharing machines. Last year, I had students work in groups so they did not have to share machines, but they ran into issues of not understanding what macros were needed to make basic file creation and execution work. This year with the per student VMs and the new Fedora SE Linux administrative GUI’s, I thought we were all set. However, the user support has changed significantly, and the new mechanism was not well documented. Plus newrole didn’t seem to work at all. I assigned my standard user separation policy where Alice is a member of two groups (or roles), so it would be natural to map that policy into different roles. Much time was wasted, but ultimately no one got the roles to work. I think next semester I’ll give up and do a class exercise. Currently, I do the MCS as a class exercise. Perhaps I’ll expand that to do a bit of policy entry. Then I’ll add a snort lab or an identity management lab.
• Written assignments - Again this year I only got one written assignment in. I just need to get thinking about this sooner. It is hard to get an interesting design assignment which everyone will have sufficient background on. Perhaps I can mine this years final projects for subsections to assign. I think that having some pure design and writing assignments is a good thing. Much of what these folks will be doing after graduation is designing and communicating designs. Perhaps I can work in a proper risk analysis exercise here.

Next fall I’m scheduled to teach the Intro to Security course again (assuming I get my contract). Last year I had a record number of 80 students. So far the number of students is much lower (40 or 50). We added another prerequisite class which may be dropping the numbers. The class will meet three times a week. In the past I’ve taught two longer sessions a week. I’m going to try and make one of the weekly meetings a more interactive class and less of a lecture. If anyone out there has ideas for security related in class exercises, I’d appreciate any and all pointers.

Been a long time since I’ve posted here. I have a few ideas on sticky notes I hope to type up in the next couple days. Got caught up in the semester and bringing our InfoSecter product to market.

We have also started an external corporate web log where Alan and I will be posting technical musings.

I’ve been working with tools and products that nominally work with network security policy for over ten years now, and the term policy is still unclear and over-used in my mind. In his security text “Computer Security: Art and Science”, Matt Bishop describes security policy to be “a statement that partitions the states of a system into a set of secure states and a set of unsecure states”. In other words, the policy defines what is means for a system to be secure, but it doesn’t describe how the system should be made secure. The policy describes the goals or intent and the mechanism defines the how. Of course policies are used from very high level organization wide Engish documents of goals and intent down to very low level configuration details like SE Linux policies.

We first used the term policy to describe our multi-device management view of our Centri firewall. Our goal was to provide a management interface that was higher level than the per device configuration and hopefully closer to what the executives had in mind when they drafted their natural language organizational security policy. This morphed into Cisco Security Policy Manager after we were acquired by Cisco. The Centri firewall targets were dropped in favor of Cisco PIX and IOS enforcement points. This “global policy” view is also presented by the Solsoft management interface. In both management products, the “policy language” is graphical. While we did raise the abstraction level from individual devices, we didn’t nearly reach an abstraction layer that would be understandable by the executive writing the organization’s natural language policy.

Morris Sloman’s group at the Imperial College London has performed a lot of work on policy language specifications. One aspect of their work is formalizing a refinement hierarchy. The policy statements at the top of the refinement are very broad, and too ill defined to be formally analyzed, but perhaps intuitively understandable to the executive’s responsible for setting the organization’s policy. At each level of the refinement hierarchy, you formalize some aspect of the policy, potentially creating multiple versions from the previous level, e.g. refining the policy for one site vs another or refining the policy for one technology (firewall) vs another (operating system). At the lowest level of the refinement hierarchy, you have policy that could be used to directly control enforcing devices. At the higher levels of the hierarchy, you have more general policies that could be used to direct a global validation of a security implementation.

In his text, Bishop identifies three points in this refinement hierarchy. One is the English or natural language policy. The others are the high level and low level formal policies. He distinguishes between these two types of formal policies by determining whether a policy statement could be picked up and used to direct the operation of a different device or different vendor’s device. A high level policy could in theory be used to direct the operation of a different device. A low level policy statement is tied to a specific device and is really more configuration than policy. He gives the type enforcement language DTEL as an example of the high level policy. This language is the parent or the grandparent of the SE Linux type enforcement policy language. While DTEL is formal and fairly detailed it operates on general concepts of subjects (or processes) and objects (or files), so one could easily see how a specific DTEL policy could direct both firewall operation and operating system operation. He gives tripwire as an example of a low level policy. Tripwire uses a database of signatures of “good” versions of system binaries. Tripwire periodically checks the signatures against the binaries, and alerts the system administrator if there is a mismatch. One might want to build an organization wide database of good binaries and signatures and install that on all systems. However, due to how tripwire is implemented the database contains very low level information (timestamps and Inode #’s I believe) that prohibit such sharing.

This concept of policy as a system crossing configuration is consistent with how policy is used in the web services and identity management world. SAML statements are in essence policy that describe how individuals should be identified between administrative domains. Marianne Winslett’s group works on algorithms and frameworks to reason about formal policies that control trust. In these cases, the policies are like little contracts that are exchanged between entities that are trying to collaborate. Unlike the network security management case, the trust policies don’t control a single administrative domain. Rather the trust policies control how the base infrastructure evolves.

One concept that recurs in most uses of the term policy is that of conflict. In any non-trivial scenario there will be multiple, conflicting goals. An organization may want to provide a easy to use web interface to encourage new customers, but the organization must also ensure that their infrastructure has sufficient authentication and auditing to avoid fraudulent customers. These two goals are necessarily conflicting. At the implementation level, one group may require http communication with a particular server, but another group may need to prohibit all communication with that server to avoid potential conflict of interest contanimation.

The policy language could force the user to clarify all conflicts through ordered lists or policy priorities. Others try to implement more intelligent systems that deduce the correct conflict resolution. A lot of the autonomic management work involves creating greater intelligence and reasoning to save the user from the error prone work of detailed conflict resolution.

Our household received two gaming devices over the holidays that are capable of communicating with the outside world. We activated the communications of the Wii, and are merrily looking at Mii’s from across the planent.

This got me to thinking about alternate attack targets. Many folks are conditioned to think about protecting communication to and from their traditional computers (though most folks don’t do a very good job of this protection). Eventually everything will have an IP address and will be capable of communication and potentially being tricked into running undesirable code.

I did a quick nmap scan of the Wii’s currently assigned IP address and nothing showed up. That’s good. It should be initiating all communication. Then I did a ping, and it got a address not routeable message. Thought that was odd, so I looked at the network scan. Turns out the Wii wasn’t answering ARP requests. Seems like a very sensible solution if it is not expecting unrequested communication.

I’ll have to do some more sniffing downstream. Many ARP implementations seem to pick up ARP information from the cache based on the requestors information. But not all. So presumably the Wii stack also replies to ARP requests if they are seen during active communication. The EULA prohibts reverse engineering the network protocol, but I presume that observing network communication to understand its impact on my network environment is fair game.

So the good news is that the Nintendo folks seem to have thought at least a little bit about communication security. The bad news is that is this won’t likely be enough in the long run. Or maybe even now. There is lots of noise about Wii hacks on the web, but they seem to be the sort of hacks you do to yourself to improve your gaming experience and impress your friends. Once someone figures out the communication channel, you can “hack” others and then the real fun begins. Wii-bots anyone?

There are lots of flops to be had from the game hardware. The PS3 was the major platform contributing flops to Standford’s Folding@home effort.

I was reading through project writeups now at the end of the semester. Two were particularly interesting and fed on each other as I read them.

One project writeup was a review of the sorry state of email security. The student reviewed current technology and came to the conclusion that it isn’t just technology that is preventing encrypted and authenticated email from being used pervasively. The technology enabling both has been around for over ten years. In my view there are a couple issues. Multiple flavors of similar technology (e.g., GPG and SMIME) that have interoperability issues. And lack of motivation on the part of the providers and the end users to insist on it. The student suggested maybe legislation needs to be passed to force people to use secure email. I agree that some motivation is needed but perhaps a legislative motivation is not the most effective.

Several years back (3 or 4?) there was a big transition with the service providers to require end user clients to authenticate to the SMTP server before it would accept mail to send. I’m sure this was a hassle for the service providers and required a lot of user support to make the transition, but it quickly became essential for them to do this. Without this authentication, the service provider was an easy target to source span. As a known spam source, the services providers addresses were blocked throughout the network, making their customers complain even louder. Perhaps some other maltechnology will effectively force the adoption of secured email.

One might argue that the pressure to have secured email is reduced because spam is perceived as less of a hassle due to the improvements in spam filtering technology. However, I would argue that spam filtering technology has just changed email into an unreliable transport. Joe doesn’t respond to my mail. Does that mean Joe doesn’t like me any more? Joe is to busy? Or Joe didn’t see my mail thanks to some intervening spam blocker? For me the later case is often true.

Another student wrote a review and some simulation results of the Internet Storm Bot, the first P2P bot net. Originally bots had a hierarchical command and control structure. If you found the head, you could pretty easily disable large bot nets. It sounds like the software malengineers have learned a lot and have adopted a lot of distributed processing techniques. No longer is there a fixed hierarchy. Each node has a set of neighbors it communicates with. The set can change over time, so if one bot is destroyed, the remaining bots can adapt. This is denser communication graph, so it is more resilient. You can find more detils on the bot net from the following workshop paper , “Peer to peer botnets: overview and case study”.

The bot engineers also frequently change their communication protocols and add encryption. With encryption, they can frequently rekey. All of this makes it more difficult for the outside observer to understand what is going on. All this change potentially also makes it more difficult to keep your bots in sync. If a bot was off the net for a while, it would miss some protocol updates. When it returns, would the bot software try to bring it up to date? How long does the bot have to be down to turn stale?

This is all bad new for a interposing bot traffic detector that I’ve been thinking about. The likelihood that a deployment could stay up to date is pretty low. When operating at the border, perhaps detecting and reacting to odd behavior is your best bet. Though if the bot nets are indeed so massive (100,000’s or 1,000,000’s of devices), the amount of malignant behavior generated by each bot might not rise above the noise. The dark address space work is one approach to making malignant behavior more apparent.

The bot paper relates to the email study because the mail means of distributing the initial infection was by getting folks to click on exe’s attached to emails. One would think after all of this time, no one would click on a weird exe. But then again, if we had some sort of proper identity authentication, I would really know the attachment was from my trustworthy friend. Thus, we lead back to having secure expectations for email that are not currently based in reality.

I’m periodically struck by now much security research and development is sold by scare talk (fear, uncertainty, and death). Unless you go over the top, you don’t get the news articles, the congressional hearings, or the money. There was a recent video going around that showed security researchers messing with a SCADA system and blowing up a power substation (in a lab) using techniques that are generally known in the community. This got congressional hearings going.

A couple years back, I saw a local researcher present simulation results that showed how two or three simultaneous errors (squirrels, trees, or terrorists) could take down the greater Chicago-land power grid. But simulation results don’t make the news. Real explosions do.

I was talking with a former colleague last week about trusted OS results from a couple decades back that were strongly worded and controversial at the time. He cynically said that the researcher in question made his statements as controversial as possible to ensure that he got attention and thus funding.

In both of these cases, raising attention through showmanship brought broader attention to valid security concerns. So I suppose the ends justify the means. It is just how the game is played.

In any case, while pulling together my network security notes for the semester, I was struck by the amount of FUD in my notes. However, I think much of the concern there is valid. Or perhaps better stated, the concern is chronic. With the IP infrastructure, our problems are chronic. We can incrementally try to make our part of the world better, but ultimately there will be infections, attacks out there. There is a shared resource out there, and by sharing we are exposing ourselves to the threat. Like getting lice by sharing a hair brush.

Perhaps we could just switch over to a newly designed, better network infrastructure. But it isn’t going to happen. Not anytime soon anyway. Still waiting for IPv6 ten years later. Even in that case, there would be threats. With the shared network, you are at the mercy of the least prepared or least savvy connected entity.

No deep observation here really. Just noting that the biological analogies fit here with respect to chronic infectious diseases.

While gathering up my notes for discussing firewall technology for this semester’s course, I started thinking about newer security issues that could be handled by the border or interposition approach of traffic cleansing.

It seems like netbot cleanup would be a good candidate for such a centralized technology. Computers that are open to infection are unlikely to be controlled by technically savvy people. So a client-oriented control is unlikely to be completely successful. However, a control installed by a technical savvy service provider could scan traffic looking for characteristic bot control traffic.

Since the service provider would be unlikely to want to annoy his client presumably, this would be on a track and alert case rather than immediately cutting the customer’s net connection. On suspicion of a remote controlled computer, the service provider could offer to do a scan of the system in question to remove well known infections.

• Given the experience with my home service provider, the depth of technical expertise isn’t there
• Unless the home service providers are getting negative feedback for hosting bots, there is no upside to going through this bother. Just pissing of your customers without getting anything in return
• The service provider customer are very price conscious and would not pay for centralized security features. This might explain we home service providers (at least around here) do not offer scanning, firewalling, or spam prevention services.
• Scanning for bot control traffic may not be feasible. The volume of control traffic would be much smaller than the bot generated traffic. While historically control traffic has been sent over IRC at a fixed port, attackers are getting clever and sending commands over different ports. The specifics of the commands could be easily changed making it difficult to separate commands from real human to human communication. I don’t know enough about today’s bot technology to know how easily it could be detected.

I’m most curious about the second issue. Given what I’ve been reading about the prevalence of bot networks, I would think this would be generating a significant amount of traffic from some service providers. Aren’t they getting negative feedback from upstream service providers and peer providers? Aren’t they getting blacklisted, etc? Or is no one tying back? In the case of address spoofed bot generated traffic doing the tie back would be difficult. But I would assume that much bot generated traffic uses its real address.

I signed up for LinkedIn last week, and I have wasted too much time on it in the meantime. Pawing through pages of old acquaintances is an excellent work avoidance technique. And since it seems kind of business oriented, you don’t have to feel as guilty about wasting time.

Just found the statistics on my “network” this morning. 25% of my network is from Brazil. Since I only have 3 contacts, my network is heavily dominated by one contact (runs a doggie day care) with over 500 contacts. Thanks to her, I am within 3 degrees of nearly everyone in my geographic area.

If you read up on security testing, one of the reasons given why testing security is so hard is that security a non-functional attribute. When writing a “traditional” test suite, you are testing the presence of a functional feature, e.g. the login widget or the client/server communication protocol. The test designer can look at the function specification and generate test cases to evaluate whether each aspect of product is working as designed. A given test case is very concrete and easy to determine pass/fail conditions. For example, the functional specification might say that the login widget should only accept user names using mixed-case alpha numeric characters. In this case the test suite might have cases to try “good” login names of various lengths and login names with bad characters of various lengths. There are issues of not being able to exhaustively test all possible inputs, but with fuzzing and other statistical techniques you can cover quite a bit of space.

Some product traits are not so concretely defined, such as security, performance, and reliability. These features are usually called non-functional characteristics, and are harder for the test suite designer to systematically address. The top level security requirement is that the product/system operates securely. So what does that mean? Hopefully, there is a security architecture for the design that dives down and identifies how security affects the design. At least then there are functional aspects of the security implementation that can be tested, e.g., functional testing of the user authentication system or the link encryption mechanism.

However, while you can approximate testing the security of the system by testing the functional aspects of the system design, there is still a big space left for negative testing. The product should operate within a security policy that defines secure and insecure states. Of the system starts in a secure state, it should continue to transition into secure states. Presumably, an attacker (or other user playing outside the rules) will try to use the system outside of how it was designed and communicated via the functional specification. Again, here there may be some directed statistically techniques to push the system through an wide variety of states. Also, threat analysis can be useful to help the test designer look at the system in non-standard ways.

Software testing is a form of system validation. The term auditing is generally used when testing a particular system installation. With auditing, the audit team is responsible for determining whether the system is working as desired. In this case, the auditor is working from an organizations’ security policy rather than a product functional specification. But again, the cases of functional and non-functional features come into play.

In my current work, we are building tools that use formal network operation specifications derived from the organization’s network security policy to determine whether a security configuration is operating within spec. Originally, I thought the fact that security is a non-functional system attribute made this validation “more difficult”, but in working through the issues in the post, I see what we are doing is validating a functional approximation of the non-functional security policy. So while security is more concerned with negative results (blocked traffic) than standard network flow engineering, the type of validation is the same.

To really consider the unbounded security auditing problem, you need consider how to question the system security model. In many ways, this outside view used by penetration testers to try to exploit flaws in the infrastructure to move the system into an insecure state.

I find much self-recognition when reading Dilbert. While I enjoy technology and building things, much of the real world of engineering sadly has a very strong human element. And unfortunately not the good warm and fuzzy aspect of the human element.

There are many good and noble efforts that get co-opted by silly humans and mutated into something ridiculous. Sadly many activities that I teach about fall into this category: Software Development Process, Risk Analysis, Security Policies Development. These are all important areas, and I’m sure that many good and earnest folks have done great work in these areas. Unfortunately, I’ve run into many other folks who have made a mockery of these processes through willful ignorance or just plain stupidity. Like my experience with “Agile” programming where only the unpleasant aspects of the process were cherry-picked, e.g., daily meetings but each meeting lasting an hour rather than ten minutes. So I can really empathize with items like the Elbonian Software Process in Dilbert.

Because these are important topics, I try to keep a positive spin when teaching about say risk analysis or security policy development. I try to show how these processes can solve real problems, and only point how they can be misused. Unfortunately, by the end of the lecture it becomes all too easy to tell stupid industry stories. So perhaps jaded, sarcastic people should not be allowed to pollute the minds of young people. Or maybe I should just stop reading Dilbert.