Table of Contents
- 1. November 2012
- 2. March 2013
- 3. May 2013
- 4. June 2013
- 5. August 2013
Leanpub book, originally based on Blog posts
This book started with the Jni4net related blog posts published on my blog and eventually evolved into the book you have in your hands (or eReader).
The idea to use my blog posts was to:
- kickstart the creation of this book (and provide a number of chapters that could be further improved)
- capture the multiple experiments and learning curves that I experienced (while learning how to use Jni4net)
- provide a ‘real world’ point of view on how to learn and use Jni4net
The theme of this book is ‘Practical’ (hence the name ‘Practical Jni4net’), and what I am trying to do here is to show how to use these technologies in practical situations, specially how to apply them to solve real problems. This means that sometimes the end result is a bit messy, with lots of rabbit-holes/tangents followed (and un-followed). But that is how it works. When learning a new technology, what is really important is to experiment a lot, and see how it works for the problem we have at hand.
An open book, freely available at GitHub
My objective with this book is to share knowledge and to help others (as I have been helped by numerous other online resources)
If you are reading this on your eReader, please visit https://github.com/DinisCruz/Book_Practical_Jni4net for a full hyperlinked copy of the content AND (more importantly) the code samples used in the book.
This repository contains the full source materials that are used to create the version available at Leanpub which can be downloaded for free, or paid for using the dynamic ‘how much do you want to pay’ slider. These fees help to cover the costs of creating this book, so if you bought the book, THANK YOU, if you haven’t, then it would be great if you could help a bit (assuming you liked and learned from this book).
I’m also using the GitHub’s Issues (see here) to manage the work items, ideas and bugs/problems with the content/code-samples.
Please use it if you want to provide any ideas, fixes or contributions to this book.
Finally, note that this book is still in a ‘draft release status’ with lots of content and fixes to be added.
I can be contacted directly at firstname.lastname@example.org (note that sometimes I’m not very good with my email, so don’t be shy to resend your email/question multiple times :) )
Notes about current structure
The current version groups the posts by published month.
I’m still thinking about the best way to structure this content, and how to present it in a easy to read/consume format.
It would be great if you (the reader) could provide some feedback on the book’s structure, for example:
- if you think the book’s order should be different (or chapters renamed)
- a particular chapter is not very clear, easy-to-understand or relevant
About the Author
Dinis Cruz is a Developer and Application Security Engineer focused on how to develop secure applications. A key drive is on ‘Automating Application Security Knowledge and Workflows’ which is the main concept behind the OWASP O2 Platform and the FluentSharp APIs.
After many years (and multiple roles) Dinis is still very active at OWASP, currently leading the OWASP O2 Platform project and helping out other projects and initiatives.
After failing to scale his own security knowledge, learned Git, created security vulnerabilities in code published to production servers, delivered training to developers, and building multiple CI (Continuous Integration) environments; Dinis had the epiphany that the key to application security is “Secure Continuous Delivery: Developer’s Immediate Connection to What They’re Creating”. This ‘Immediate Connection/Feedback’ concept is deep rooted in the development of the O2 Platform, and is something that will keep Dinis busy for many years.
Here are the changes made (per version):
v0.22 (Jan 2016)
- Major improvements on editor-ui where it now renders a local site using express and markdown
- Changed intro, removed ‘Making Java, .NET and C++ apps work together’ post (which is now part of the intro)
- Major refactor and clean up of content files:
- Each month has it own folder which contains the respective images
- Checked/fixed all images, and updated links to referenced Pdfs
- All files were renamed to allow easy editing and maintenance
- Significant content fixes and refactoring
- Reached 95% book completion (waiting for feedback before marking it as complete)
v0.21 (Dec 2015)
- Started text refactoring in order to get better structure for book generation and book editing
v0.20 (April 2015)
- Setup leanpub to use Github instead of Dropbox
- updated content: Introduction
v0.16 (Mar 2014)
- First release of book with raw import from blogger posts (no formatting or editing done)
- Created Git repo on local dropbox sync folder
- created github repository for this book: https://github.com/DinisCruz/Book_Practical_Jni4Net
- Added this change log
In November 2012 I found a way to make Java and .Net applications work together.
That is something that I’ve always wanted to be able to do, since it would allow the use of O2’s REPL in Java application.
The the key technology innovations that were required to make this happen were:
- 1) Side-by-site JVM/CLR execution and programming (in same process) - This is the ability provided by the amazing http://jni4net.sourceforge.net/ FOSS project which creates bridges between Java and .Net (see this research paper for the main concept between .Net/Java bridges http://views.cs.up.ac.za/vdata/Views-CCPE.pdf )
- 2) Injection of Managed/Unmanaged controls into another process’s controls/windows - This the capability of the O2 Platform that allows the easy injection of .NET dlls into other non managed process.
Note that this is still a massive hack, but it works :)
It would be great if there was a more official way to make this happen.
This book contains my Jni4Net experiments and research.
1. November 2012
- C# REPL a java process (ZAP Proxy)
- Controlling OWASP ZAP remotely (via Java BeanShell REPL in .Net)
- Writing and Invoking O2 Methods from Java and Eclipse
C# REPL a java process (ZAP Proxy)
After I discovered the Jni4Net FOSS project which provides the foundation blocks to create a Java bridge to .Net (and vice-versa).
To try Jni4Net, and see if it was really possible to have .Net and Java code running on the same process (with the CLR and JVM being able to invoke each other’s methods), I decided to see if I could connect the O2 Platform with the OWASP ZAP project (with both running on the same process)
My first step was to use the Injecting C# DLLs into Managed (C#) and Unmanaged (C++) processes O2 capabilities to inject an C# REPL into the ZAP process.
And here they are in action:
- the foreground window is the (CLR based) C# REPL
- the background window is the (JVM based) ZAP process.
After some experiments, I was able to use Jni4Net to create an mini .Net tool (based on the the Util - Java Decompiler (JAD based) v1.0.exe ) that used Jni4Net’s direct access to the JVM to show (and browse) the the currently loaded *.jar files (based on data retrieved from the JVMs ClassLoader).
For example here are some ZAP classes, and the decompiled main method:
For more details on how these PoCs were created, take a look at https://bintray.com/artifact/download/dinis-cruz/Books/Using-Jni4Net-Part-1.pdf
Controlling OWASP ZAP remotely (via Java BeanShell REPL in .Net)
Once I was able to use C# REPL in java process I was able to use the ZAP BeanShell support to start controlling and manipulating ZAP’s GUI from the .Net C# REPL.
Here is an example of adding a new ‘Site’ (i.e. a TreeNode) to the main ZAP ‘Site’s window (i.e. TreeView)
For more details on how this PoC was created, take a look at https://bintray.com/artifact/download/dinis-cruz/Books/Using-Jni4Net-Part-2.pdf
Writing and Invoking O2 Methods from Java and Eclipse
After using C# REPL in java process and controlling OWASP ZAP remotely, the next step was to see if we could consume (and code) the .NET APIs from Java.
And again Jni4Net really worked!
Here is a .NET WinForms control, coded and executed from the (Eclipse written) Java code:
Once we could create *.jar files, it was a small step to create an Eclipse plugin that would load up a CLR and popup a C# based Form.
Or more interestingly an C# REPL editor (running in the same process as Eclipse):
For more details on how these PoCs were created, take a look at https://bintray.com/artifact/download/dinis-cruz/Books/Using-Jni4Net-Part-3.pdf
2. March 2013
- Invoking an OWASP AppSensor Java method from .NET C# REPL
- Invoking Java BeanShell from .Net CLR
- Loading OWASP ESAPI jar and its dependencies
- Getting list of Jars loaded in SystemClassLoader
- Java BeanShell REPL v1.0.exe
Invoking an OWASP AppSensor Java method from .NET C# REPL (using Jni4Net)
On the topic of AppSensor, you might find the code snippet below interesting.
- load the AppSensor jar in a new class loader,
- access/view its classes in a GUI
- create an instance of org.owasp.appsensor.trendmonitoring.TrendEvent
- execute the getTime method
Note that the AppSensor code is running on the Java’s JVM (loaded in the same process as the .Net’s CLR)
The code is still in very rough status, but it works :)
Here (screenshot below) is what it looks like: * note the Date value in the output window (bottom right) which is the result (shown as a .NET String) of the Java getTime method * on top you have a view into the AppSensor’s classes, methods, parameters, return type * on the bottom left you can see the written C# REPL script
Here is the code:
note: this code sample was running on the x64 version of the O2 Platform (because the default JavaHome in my dev VM points to a x64 Java version)
Invoking Java BeanShell from .Net CLR
Here is a very rough PoC of how I was able to execute a JavaBean shell script from inside the O2 Platform (with the java code executed under a JVM)
Executing “return 2+2;” as a java beanshell comand (see result on the bottom right Output pane)
Loading OWASP ESAPI jar and its dependencies
Here is a pretty cool PoC where I was able to load an jar file and its dependencies into an ‘Jni4Net created’ JVM
Using the system class loader
This is the ESAPI class we want to load (the code below returns null)
Using the technique shown in Adding files to java classpath at runtime - Stack Overflow , we can add the esapi jar into the sysLoader class path
which will now work:
If we now try to load all classes in the ESAPI jar, we will get 172 classes and a number of class load errors (due to missing jars references)
Let’s refactor the code and create the addJarToSystemClassLoader method
And load one of the dependencies (note the increased number of classes loaded (183)):
And now if we load all jars in the libs folder (all 30 of them), we will get 197 classes and no load errors
Here are the jars added to the classpath:
Here is the code (shown above) that loaded all esapi classes into the system class path:
Once these classes are loaded we can use the tool shown in C# REPL a java process (ZAP Proxy) to browse them and view its source code:
We can now create instances of ESAPI using reflection.
One problem to solve is the need to define where the ESAPI.properties file is:
The code above will throw an error on line 439 of the DefaultSecurityConfiguration file
Using a separate class loader
Here is a script that loads 172 classes from the ESAPI jar
This version will load 183 classes since we are also loading the servlet-api
Getting list of Jars loaded in SystemClassLoader
I just created a couple extension methods for Jni4Net that allow (amongst other things) the listing of the jars currently loaded in the SystemClassLoader (see API_Jni4Net.cs for the code of these .NET Extension Methods)
The objective is to simplify the use of Jni4Net, and to hide the complexity in consuming Java code from .NET.
Here are a couple examples of these Extension Methods in action:
When you create a simple Jni4Net bridge there is only one jar loader:
Next lets load a jar dynamically and see it appear in the list of loaded jars
Note that once the jar is loaded we can access its classes:
Create instances and invoke methods:
Here is the script show above:
Finally we can also load/add to the system class path, entire folders with jars. For example all ESAPI dependencies:
Java BeanShell REPL v1.0.exe
Using the technique shown in Invoking Java BeanShell from .Net CLR, here is REPL that allows the quick execution of Java BeanShell command in a C# GUI
You can download this stand-alone O2 tool from: Util - Jni4Net - Java BeanShell REPL v1.0.exe
This is what the default GUI looks like:
If you expand the Console Out panel, you can see the Jni4Net initialization messages:
There are a number of code samples included:
Which will be auto executed on selection:
This one for example:
will create a Java button:
which of course can be modified and executed:
Also included is a REPL menu
That allows the scripting of the current Form:
like for example: changing its title and making all controls pink
or inject a WebBrowser (on the left) with the Jni4Net website:
The other REPL menu item:
Allows the scripting and visualization of the Jni4Net object:
For reference here is the script (also available at O2.Platform.Scripts\3rdPartyJni4NetUtil - Jni4Net - Java BeanShell REPL.h2) that created this GUI (a bit messy and in need for a good dose of refactoring)):
3. May 2013
- First execution of ESAPI.jar Encoder methods from O2’s C# REPL
First execution of ESAPI.jar Encoder methods from O2’s C# REPL
In April 2013 (when preparing to do the OWASP AppSensor and O2 Platform at Security B-Sides London workshop with colin), I achieved something that I have been trying to do for a while:
- Use the O2 Platform’s C# REPL environment to programatically access (in real time) the ESAPI jar apis and methods.
- Consume AppSensor’s jar directly from a .NET app (in that case TeamMentor)
When I was integrating AppSensor (java) with TeamMentor (.NET) I needed to solve the the problem described in Loading OWASP ESAPI jar and its dependencies from C# since AppSensor has an hard dependency on ESAPI (i.e. without ESAPI, AppSensor doesn’t work)
After a bit of research and coding, here is the first code sample that shows ESAPI being consumed from O2 (via Jni4Net)
Note how the ESAPI Encoder’s encodeForHTML method is being executed:
And displayed in the Output window:
To to clarify, although this code is being triggered and executed from a .Net GUI and CLR, the actual code that is doing the encoding is the one from http://owasp-esapi-java.googlecode.com/svn/trunk_doc/latest/org/owasp/esapi/Encoder.html which is running running on a JVM inside the same process as the CLR (the magic sauce is provided by the amazing Jni4Net)
Which is exactly what I wanted :)
I wonder if we can also execute the current ESAPI unit test the same way? For example, now that we have the capability to see what the encodings output of the Java version of ESAPI looks like, we can try to compare it with the ESAPI .NET methods (since they should match right?)
Here is code sample shown in above:
Here is the current folder set-up and jars loaded:
A key to make it work is the fact that the ESAPI.properties file is in the same path as the ESAPI.jar
4. June 2013
- Another step in the use of ESAPI and AppSensor Jars from .Net/C#
Another step in the use of ESAPI and AppSensor Jars from .Net/C
At the OWASP EU Tour London Chapter event meeting I presented the next step of my research on using ESAPI and AppSensor inside a .NET application like TeamMentor (using Jni4Net to allow the JVM to work side by side with the CLR).
The source code of the demo I presented is posted to the github.com:DinisCruz/TeamMentor_3_3_AppSensor repo, and this post shows a number of screenshots of what is in there.
I used TeamMentor’s TBot C# and AngularJS pages to create the prototypes (since it is very easy and fast to code in that enviroment)
The pages were added to the main TBot ** control panel, in 3 new sections: **AppSensor, AppSensor/ESAPI and AppSensor/JVM:
Let’s look at all of them and see what they do.
Shows the Properties of the current JVM, and is a good first script to run (since it shows that the Jni4Net CLR to JVM bridge is correctly set up)
Jars In Class Path
This one shows the Jar’s currently loaded and some details about the loaded classes
The image above shows that there is only one jar loaded at start (jni4net.j-0.8.6.0.jar) and below is what it looks after the Setup AppSensor Tbot page is executed
This will load up the AppSensor Jars and perform a simple test to see if one of the expected classes can be loaded
View ESAPI Encodings
Once we have the ESAPI loaded we can open up this page that shows what all the ESAPI encodings looks like
For example change the text on the left and click on of the ‘encodeFor…‘ buttons
Shows the currently registered logs
To help to create a new log entry, this page provides a link to:
Create AppSensor Exception
which looks like this:
This page (for testing) allows the use of the ex querystring parameter to create a new AppSensor log message
and clicking on View AppSensor Logs, which show details of the log:
5. August 2013
- Consuming a forked version of Jni4Net
- Compiling Jni4Net from Source
- Fixing the ‘Attempted to read or write protected memory’ issue
Consuming a forked version of Jni4Net
An O2 user was having some issues with my Jni4Net scripts, and after some research, I realized that the problem existed inside the current published jni4net.n-0.8.6.0.dll:
The problem is that with the 0.8.6 version we get an ‘Attempted to read or write protected memory’ error when trying to connect to an existing Java process we previously injected O2 into.
The solution is to use the latest version of Jni4Net, which is not available for direct download, but its code it at: http://jni4net.googlecode.com/svn jni4Net
After getting the compilation and generation process right (see Dealing with ‘Attempted to read or write protected memory), I had this folder:
I was then able to write these scripts (which worked):
1) start ZAP and inject O2 into the new process:
2) hook VisualStudio to that process (assumes there is only one java.exe process)
3) create bridge and output java properties (inside the ZAP REPL)
4) Executed PoC - Jni4Net - Classes, Methods, Fields in target.h2 script (inside ZAP process)
which listed all loaded java classes (and its decompiled code):
To make this work (and deal with the ‘Attempted to read or write protected memory’ prob) I had to copy the jni4net.j-0.8.7.0.jar to the ZAP default class path and manually skip a bit of code when connected to VisualStudio
Compiling Jni4Net from Source
After using the git svn clone -s http://jni4net.googlecode.com/svn jni4Net command, to create a local clone of the SVN repo:
I started by opening the jni4net.sln file in VS 2010.
But on first compile I had this error:
which was created by the missing selvin.exportdllattribute-0.2.6.0.dll
which looks like should be here:
let’s take a look at the v0-8-generics branch
Since it is also not there, I just downloaded it from:
… and saved it to:
… and updated the references
Now the code compiles, but we get this error:
… which is caused by this post build command:
… and is resolved by downloading this file:
… into this folder:
I was able to get it to compile by changing the selvin.exportdll-0.2.6.0.exe to selvin.exportdll-0.2.5.0.exe (couldn’t find the 0.2.6.0 version)
Next there was a bunch of NUnit errors, which were resolved using
Now we have a clean compilation (with only a bunch of warnings)
But, they all fail to execute:
This was a good sign that I should look for a readme.txt, which I found in the repo root (ie. RTFM when everything else fails :) )
From that file I got the clue to run the loadTools.cmd script:
… which downloads the missing files into the lib folder:
Since we now have the 0.2.6.0 versions, I updated back the VS solution (and removed the nUnit package from Nuget)
Then I executed
… this took a while, and although most looked ok:
… the last one failed:
That said, it looked like the required files seemed to have been created:
And the Unit tests now pass:
I didn’t like the fact that we had that build error, so I tracked the problem to a missing NUnit installation:
Since I’m on a x64 VM and the path is hard-coded in the Maven script, I had to copy the installed nunit files into the expected folder:
… which didn’t work
The problem was caused by the fact that I’m currently using an x64 JDK:
Forcing a x86 (32bit) version of the JDK will do the trick
Since the tests expect to find Nunit in the (x86) folder:
We need to rename the NUnit-2.5.8 folder to the expected value:
Finally we have a successful complete build:
… with all created files placed inside the target folder:
Fixing the ‘Attempted to read or write protected memory’ issue
This error happened when running a java process (like ZAP):
…with the O2 Platform REPL injected into:
… and executing PoC - Jni4Net - List JavaProperties.h2 scrit, resulted in an unmanaged error (which is always a worse case scenario in the .NET Interop world)
As the (simpler script shows), the error happens on the CreateJVM invocation
My next step was to create a local build of Jni4Net (see Compiling Jni4Net from Source) and to use it to attach into an existing running ZAP with an O2 REPL injected.
Here is the moment when I have VisualStudio hooked, the script compiled (in the ZAP REPL), the jni4net dll loaded (with Symbols) in VisualStudio and a breakpoint on the CreateJVM method:
Here is where I think the problem exists:
The args object (created via JNI) seems to be empty.
It is then assigned to the value of the Jni4Net class path string
The args object is used here:
And inside that method, if I let args to be used, we will get the ‘Attempted to read or write protected memory’ error
But, if I change the execution path (manually) and set it to the line below (where no args is passed)
Then the execution will be ok (note that the path with the &initArgs would had thrown an ‘Attempted to read or write protected memory’ by now):
The only problem with this approach (which is basically not adding the classpath clue to the current JVM) is that unless we manually add the jni4net jar to the target app, we will get an exception here:
i.e. br.handle would be 0 (meaning that the net.sf.jnin4net.Bridge class could not be found)
To fix it, I added the Patch_IgnoreArgsInAttach property
… which is then used here (to allow the selection of the path that works)
I also changed the assembly name (so it is easy to track its use):
After the compilation I copied it to the main target folder:
To test it , I used the script
… to start ZAP with a REPL, where I could execute OK:
Final step is to create a zip of the bin folder
Put it on a public available location like DropBox
Update the installer API to use that version:
Update the API_Jni4Net.cs to use the jni4net.n-0.8.7.0_Patched.dll assembly (as seen below on first compile the referenced will be downloaded)
… once the compilation works:
I was able to to run Jni4Net in the main O2 Platform process
…and on the ‘injected into’ java processes:
Pushing changes to GitHub
Once the fix was working ok I added a remote to the local repo:
… push it
… and confirm that the commits I did locally (with the patch)
… where pushed successfully: