Thursday, 17 September 2009

Fortify hands-on demo/session at forthcoming OWASP Northern Virginia Chapter

So, the adventurous OWASP Virgina Chapter (lead by the uncompromising John Steven) are going into uncharted-OWASP waters in their next chapter meeting.

You can read more about it on the chapter home page on their [Owasp-wash_dc_va] OWASP Session - Fortify 360 - Thursday, September 17, 2009 mailing list announcement or at the Secure Coding Mailing list

Basically what they are doing is allowing a vendor (Fortify) to come to an OWASP meeting and present their product! Shock Horror!!! Doesn't this break OWASP values, principles and independence!!!

Well, it depends :)

OWASP is not Anti-Vendor! In fact most of OWASP members and users are either direct connected to a vendor or use vendor's products/services (disclosure one of my contacts is with Ounce labs (now IBM)). In fact vendor presentations at OWASP happen ALL the time (see for example this presentation delivered at the last OWASP London chapter Using Surrogates to Protect from Application Data Breach ).

The issue is not IF OWASP should have 'vendor' presentations but HOW we do them. My view is that as long as the 'snake oil & marketing' content is kept under control, what is presented is an 'accurate' representation of that technology and there is interest of the OWASP community in it, then it is OK.

The fear is that OWASP become an 'vendor driven' organization and becomes 'infiltrated' with people who have direct & short-term commercial priorities. The good news is that I think OWASP has a long and ingrained tradition of 'keeping the vendors under control' and as we grow we need to create 'environments' where the vendors can show where they add value in a way that is compatible with OWASPs values and principle.

And in my view, John is trying to create this environment using a 'real-world' case study (btw, this is what I love about OWASP, our leaders have the ability to be proactive and creative (we just need to make sure they are going on the right direction :) ))

So, back to the subject at hand, here are a couple points and ideas about allowing vendors to provide 'hands-on sessions at OWASP Chapters and conferences' (I would like to see at the end of this thread a nice list of 'rules of engagement' for other chapters/conferences that want to organize similar events):

1) this is not a new idea, we have had many numerous talks in the past about helping to create at OWASP conferences an 'open & independent lab environment where people can try technology', and in fact I organized a while back a bake-off between WAF vendors in London (see London_Chapter_WAF_event),
2) The vendor should provide unrestricted and uncontrolled access to the technology to the participants,
3) On the other hand, since the value derived from these tools is usually very dependent on them being used by 'experienced users' and the fact that there is a section of the OWASP community that is very technical (& historically very skeptical about the REAL value that these tools can provide), the vendor (ideally) in partnership with an independent service provider, should also show how their tool is used in real world scenarios by its users,
4) The attendees should be allowed to take with them an evaluation version of the product without having to provide any information in return (business cards, names, mobile phones, social security numbers, bank account details, etc... :) )
5) Pending technologically or licensing problems, the vendor should provide a VMWare/VirtualPC/XEN/OWASP_Live_CD image containing everything needed to evaluate this technology (for windows, I think we could use 30/60/90 day evaluation versions of the required OS)
6) Pending bandwidth or logistical issues the event should be broadcasted live and remote users should be give access to virtual images
7) Pending technological or logistical issues the event should be recorded in video/audio and made available to OWASP users
8) Final and very important, the final decision if one of these events is 'successful and respects OWASP's values and principle', should be made by the local OWASP 'non-vendor' members (i.e. people from local companies that are trying to buy, develop or maintain secure web applications). What I found in the past, is that the threshold for 'vendor pitches' is very dependent on geographical locations (i.e. the same presentation in NYC and in Milan will have very different reviews (and sometimes the non-US chapters tend to be much more 'vendor' friendly)). So I would look at the local chapter (users and leader(s) ) for guidance about the event's outcomes.

If this is popular, we should make these activities/events into an 'OWASP Project' since we will need to keep a tight control on these rules and ensure that this doesn't get abused.

BUT, if we get this right, we will be able to leverage much more the energy/motivation that the vendors have in promoting their products, with the energy/motivation of the consulting companies that know how to use those products, and (MORE IMPORTANTLY OF ALL) with the needs, requirements and issues that the users/clients have.

What do you think? This is a though issue, but it is HAPPENING, so we might as well agree on the 'rules of engagement'

From the current description of the 'Fortify at Virgina chapter' event, I think they meet just about all the items I propose. Any comments?

Dinis Cruz


diniscruz said...

(also sent to the owasp-leaders list)

Just consolidating some of the comments I seen on this list or received privately.

Here are a couple more items/ideas for the 'rules-of-engagement'

A) On the vendor behavior:

1) No calls to action
2) No talk about price or licensing
3) Allow competitors to join the talk and give counterpoints (like the full week it takes to get Fortify to run against a large codebase - even longer for Ounce)
4) Preferably having someone OTHER than the vendor do the talk may help.

B) I wouldn't count on every chapter leader to organize such a thing. Maybe it should be coordinated with the OWASP board? Alternatively, maybe a chapter leader should get 5 leaders to say he will do right to proceed?

C) Publish the detailed content in advance, or at least communicate it to the chapter leader. I actually require every speaker who work for a vendor to submit the slides before the meeting to inspect them for over commercialization.

D) Require specific speakers. It's the speaker who makes the presentation. Just having a "company" present, sending an available sales engineer has a huge potential of being a bummer. If you get an engineer from dev, or at least a product manager who can shed some light on future directions you are better off.

E) Ensure 'sender email' usage is consistent: "...My little comment is that I am wondering why Eric Dalci used his Yahoo email address yesterday to announce the event on the Secure Code
Mailing List (SC-L) while previously he has used his Cigital email address on the Virginia chapter mail list (e.g. Perhaps splitting hairs but some might perceive it to be sneaky...."

I (Dinis) am pretty sure i've read a variation somewhere of the next three ideas, but it is getting too late, so here they are they way I remember them :)

F) all non-vendor participants need to disclose their relationship(s) with the vendor(s). For example I received the comment today "...Cigital owns most of Fortify..." , is this true? I don't know! (John might should be able to clarify this), but the key point is that the audience should know this information.

Once this information is disclosed, the audience can independently make up on their minds on how 'independent' the 'independent security consultant' really is :) .... (yes ... on this last independent paragraph I did made an effort to independently use as many as possible variations of the word 'independent' :) )

G) Don't have a vendor employee participating on the presentations at ALL!!. Ideally the presenters (& people helping) should be as removed as possible from the vendor. I think security consultancy companies like Cigital are a good compromise (as long as they behave :) (I trust John Steven on this one)), but it would be great if the people driving this would be the ACTUAL users of these products (who tend to have a much more pragmatic view of the value provided by them AND how to work around the limitations of those projects)

H) do a 'multi-vendor' event where the multiple products are shown in action with the same target apps (from easy to hard) and test environments (btw, this is what I (Dinis) tried to do with the London WAF bake-off a couple years ago) . Note that although this option is the most interesting of them all, it is the most complicated to execute from a logistically point of view (and the one that needs to resources from OWASP to organize it properly)

Keep the comments coming, since I think we are almost there :)


המועצה לצרחנות said...

Even with no >blunt< "snake oil & marketing" content, vendors still get the chance to sell their product this way. You are basically giving vendors a "booth" to show their product.

I think this is a bit shady...