Ionic – The best cross-platform mobile framework

I’ve been patiently waiting for years for a solid cross-platform mobile framework to emerge, and finally one has, the Ionic Framework.  Cordova/PhoneGap started things off by bridging the gap between web views and native code and then Ionic took it too a new level with Angular and beyond with Ionic services.  Ionic services include push notifications, deploy, analytics, and package.  They have even created a marketplace to share and sell themes, templates, components, etc.  Ionic deploy is revolutionary allowing developers to update their app without resubmitting it to the app stores for approval.  I sure hope the app stores don’t put a stop to it.   Ionic package is a service that will package up Android and iOS builds of your app.  This is a great help when you don’t have a Mac machine, but you still want to build iOS apps.

The Ionic framework isn’t ideal for games or high performance apps, but it is great for the majority of business apps.  Ionic has the goal of being the ‘WordPress’ of the mobile world and they are well on their way.  They are well funded and have great vision and have showed great execution.  They keep the barrier of entry low by providing free tiers of their services.  Angular has become the leading javascript framework so it provides a nice, neat, and familiar foundation to build apps with HTML, CSS, and JavaScript.  The majority of modern software engineers already know these technologies.   Angular 2 will be coming out and Ionic is already working on integrating it into their platform.

You can’t go wrong with learning Ionic, because I believe it is a framework and platform that will stick around for a long time.  I need to learn it more in depth and produce more apps.  Hopefully I can share some Ionic tutorials here in the future from what I learn and apps I am able to develop with it.

Image Proxy using Jersey and HttpClient

Here’s the Jersey Endpoint:

    @GET
    @Path("imageproxy")
    @Produces("image/png")
    public Response imageproxy(@QueryParam("url") String url) {
        byte[] result = null;
        try {
             result = ImageHelper.getUrlBinary(url);
        } catch(Exception ex) {
            logger.error("Error proxying image", ex);
        }
        if(result != null) {
            return Response.ok(new ByteArrayInputStream(result)).build();
        } else {
            return Response.noContent().build();
        }
    }

Here’s the HttpClient code:

import org.apache.commons.io.IOUtils;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.log4j.Logger;
import java.net.URI;

public class ImageHelper {
    private static Logger logger = Logger.getLogger(ImageHelper.class);
    public static byte[] getUrlBinary(String url) {
        byte[] result = null;
        try {
            URI uri = new URI(url);
            HttpClient client = new DefaultHttpClient();
            HttpGet httpGet = new HttpGet(uri);
            HttpResponse response = client.execute(httpGet);
            result = IOUtils.toByteArray(response.getEntity().getContent());
        } catch(Exception ex) {
            logger.error("Error getting binary from: " + url, ex);
        }
        return result;
    }
}

Java – TAXII – Collection Management Endpoint

As a followup to my last post, here’s a partial implementation of the collection management endpoint for a TAXII server. I added error handling via the TAXII status message response too.

    @POST
    @Path("collection")
    @Consumes (MediaType.APPLICATION_XML)
    @Produces (MediaType.APPLICATION_XML)
    public Response collection(@Context HttpServletRequest request, String x) {
        CollectionInformationRequest collectionRequest = null;
        SubscriptionManagementRequest subscriptionRequest = null;
        try {
            //printHeaders(request);

            System.out.println("---------- Request:");
            Object requestObject = getRequestObject(x);
            if(requestObject instanceof CollectionInformationRequest) {
                collectionRequest = (CollectionInformationRequest) requestObject;
            } else if(requestObject instanceof SubscriptionManagementRequest) {
                subscriptionRequest = (SubscriptionManagementRequest) requestObject;
            } else {
                throw new Exception("Unsupported request type");
            }
            
            System.out.println("---------- Response:");
            if(collectionRequest != null) {
                System.out.println(toXml(collectionRequest));

                List collections = new ArrayList();
                collections.add(factory.createCollectionRecordType()
                        .withAvailable(true)
                        .withCollectionType(CollectionTypeEnum.DATA_FEED)
                        .withCollectionName("default")
                        .withDescription("Default data set description")
                        .withPollingServices(factory.createServiceContactInfoType()
                                .withAddress("/poll")
                                .withMessageBindings(Versions.VID_TAXII_XML_11)
                                .withProtocolBinding(Versions.VID_TAXII_HTTP_10)
                        )
                        .withSubscriptionServices(factory.createServiceContactInfoType()
                                .withAddress("/collection")
                                .withMessageBindings(Versions.VID_TAXII_XML_11)
                                .withProtocolBinding(Versions.VID_TAXII_HTTP_10)
                        )
                        .withContentBindings(factory.createContentBindingIDType().withBindingId(ContentBindings.CB_STIX_XML_111))
                );

                CollectionInformationResponse collectionResponse = factory.createCollectionInformationResponse()
                        .withInResponseTo(collectionRequest.getMessageId())
                        .withMessageId(MessageHelper.generateMessageId())
                        .withCollections(collections);

                String responseString = toXml(collectionResponse);
                System.out.println(taxiiXml.marshalToString(collectionResponse, true));

                return generateResponse(responseString, request);
            } else {
                System.out.println(toXml(subscriptionRequest));
                
                String subscriptionId = subscriptionRequest.getSubscriptionID(); // Should be null on a subscribe
                CollectionActionEnum action = subscriptionRequest.getAction();
                PushParameterType pushLocation = subscriptionRequest.getPushParameters();
                
                // Gather type, query, content bindings so we know what kind of delivery they want
                // Store/update their subscription based on the desired action.  
                // Pause/Resume are tricky because you need to pick up where they paused and send what they missed
                
                SubscriptionManagementRequest subscriptionResponse = factory.createSubscriptionManagementRequest()
                        .withMessageId(MessageHelper.generateMessageId())
                        .withSubscriptionID(subscriptionId)
                        .withCollectionName(subscriptionRequest.getCollectionName())
                        .withAction(subscriptionRequest.getAction());
                
                String responseString = toXml(subscriptionResponse);
                return generateResponse(responseString, request);
            }
        } catch(Exception ex) {
            return handleError(ex, request, collectionRequest);
        }
    }

    private Response handleError(Exception ex, HttpServletRequest httpRequest, RequestMessageType taxiiRequest) {
        ex.printStackTrace();
        try {
            StatusMessage status = factory.createStatusMessage()
                    .withMessage("Error: " + ex.getMessage())
                    .withInResponseTo(taxiiRequest != null ? taxiiRequest.getMessageId() : null)
                    .withMessageId(MessageHelper.generateMessageId())
                    .withStatusType(StatusTypeEnum.FAILURE.value());
            String responseString = toXml(status);
            return generateResponse(responseString, httpRequest);
        } catch(Exception e) {
            e.printStackTrace();
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR).build(); 
        }
    }

Java TAXII Server Implementation

TAXII is a protocol standard for sharing security threat information between systems.  TAXII operates over HTTP/HTTPS and uses a specific XML schema for the message payloads.  Typically STIX is used as the main content of the messages.  TAXII is slowly being adopted by various organizations, particularly the banking industry.  The folks backing TAXII have provided YETI, a basic Python/Django TAXII server implementation and a Python client as well.  They also have a Java client which provides all the JAXB XML mappings for the specification.  In most enterprise environments, Python will not be ideal.  I decided to take the TAXII Java Client code and produce a simple Java Jersey (JAX-RS) TAXII server implementation of the DISCOVERY and POLL endpoints only. There are lots of println’s to show what is going on.  I built the java-taxii  and stix-binding code into a jars and loaded them into a local Maven repo to have available.

Dependencies: com.sun.jersey:jersey-bundle:1.17, javax.servlet:servlet-api:2.4, org.mitre.stix:stix-bindings:1.0 and org.mitre.taxii:java-taxii-all:1.0

Avalanche/SoltraEdge is a nice implementation of TAXII with a great UI, but licensing of the community edition is very limited in terms of usage and customization/modification.  It is developed in Python/Django as well and is most likely a spin-off from YETI.  Their website is incomplete, so it is hard to know what their commercial offerings are.  If you already have your security threat data stored somewhere, you’ll want to be able to customize your own TAXII interface in front of this data store.  Even YETI assumes you want to store incoming data in its own SQLITE database and serve it up from there.

The protocol is well documented, so it is easy to understand what each field is for so you can abide by the specifications.  The INBOX and COLLECTION_MANAGEMENT endpoints are more complex than the POLL and DISCOVERY endpoints, but can be done in a similar fashion.

Follow-up post with a sample/starter collection management endpoint: Java Taxii Collection Management Endpoint

import java.io.*;
import java.math.BigInteger;
import java.net.URI;
import java.util.*;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.*;
import javax.ws.rs.core.*;
import javax.xml.bind.*;
import javax.xml.datatype.*;
import org.mitre.taxii.*;
import org.mitre.taxii.client.HttpClient;
import org.mitre.taxii.messages.TaxiiXml;
import org.mitre.taxii.messages.xml11.*;

@Path("/")
public class TestEndpoints {

    /* Example request headers
            User-Agent: java-taxii.httpclient
            content-type: application/xml
            accept: application/xml
            x-taxii-accept: urn:taxii.mitre.org:message:xml:1.1
            x-taxii-content-type: urn:taxii.mitre.org:message:xml:1.1
            x-taxii-services: urn:taxii.mitre.org:services:1.1
            x-taxii-protocol: urn:taxii.mitre.org:protocol:http:1.0
    */
    
    // Are these thread-safe???
    private ObjectFactory factory = new ObjectFactory();
    private TaxiiXmlFactory txf = new TaxiiXmlFactory();
    private TaxiiXml taxiiXml = txf.createTaxiiXml();
    
    @POST
    @Path("discovery")
    @Produces (MediaType.APPLICATION_XML)
    @Consumes (MediaType.APPLICATION_XML)
    public Response discovery(@Context HttpServletRequest request, String x) {
        try {
            printHeaders(request);

            System.out.println("---------- Request:");
            DiscoveryRequest discoveryRequest = (DiscoveryRequest) getRequestObject(x);
            System.out.println(toXml(discoveryRequest));

            System.out.println("---------- Response:");
            List services = new ArrayList<>();
            services.add(factory.createServiceInstanceType()
                    .withServiceType(ServiceTypeEnum.POLL)
                    .withAddress("/poll")
                    .withAvailable(true)
                    .withProtocolBinding(Versions.VID_TAXII_HTTP_10)
                    .withServiceVersion(Versions.VID_TAXII_SERVICES_11)
                    .withMessageBindings(Versions.VID_TAXII_XML_11)
                    .withMessage("Super awesome data comes from this service")
                    .withContentBindings(factory.createContentBindingIDType().withBindingId(ContentBindings.CB_STIX_XML_111))
            );
            
            DiscoveryResponse discoveryResponse = factory.createDiscoveryResponse()
                    .withInResponseTo(discoveryRequest.getMessageId())
                    .withMessageId(MessageHelper.generateMessageId())
                    .withServiceInstances(services);
            
            String responseString = toXml(discoveryResponse);
            System.out.println(taxiiXml.marshalToString(discoveryResponse, true));
            
            return generateResponse(responseString, request);
        } catch(Exception ex) {
            ex.printStackTrace();
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR).build();
        }
    }
    
    @POST
    @Path("poll")
    @Produces (MediaType.APPLICATION_XML)
    @Consumes (MediaType.APPLICATION_XML)
    public Response poll(@Context HttpServletRequest request, String x) {
        try {
            printHeaders(request);

            System.out.println("---------- Request:");
            PollRequest pollRequest = (PollRequest) getRequestObject(x);
            System.out.println(toXml(pollRequest));
            String type = pollRequest.getPollParameters().getResponseType().equals(ResponseTypeEnum.FULL) ? "FULL" : "COUNT ONLY";
            System.out.println("Response Type: " + type);
            System.out.println("Collection: " + pollRequest.getCollectionName());
            System.out.println("Start Time: " + (pollRequest.getExclusiveBeginTimestamp() != null ? pollRequest.getExclusiveBeginTimestamp().toXMLFormat() : "(none)"));
            System.out.println("End Time: " + (pollRequest.getInclusiveEndTimestamp() != null ? pollRequest.getInclusiveEndTimestamp().toXMLFormat() : "(none)"));

            System.out.println("---------- Response:");
            PollResponse pollResponse = factory.createPollResponse()
                    .withInResponseTo(pollRequest.getMessageId())
                    .withMessageId(MessageHelper.generateMessageId())
                    .withCollectionName(pollRequest.getCollectionName())
                    .withRecordCount(factory.createRecordCountType().withValue(BigInteger.valueOf(9999)).withPartialCount(false)) 
                    .withExclusiveBeginTimestamp(pollRequest.getExclusiveBeginTimestamp())
                    .withInclusiveEndTimestamp(pollRequest.getInclusiveEndTimestamp())
                    .withContentBlocks(
                            factory.createContentBlock()
                                    .withContentBinding(factory.createContentInstanceType().withBindingId(ContentBindings.CB_STIX_XML_111))
                                    .withContent(factory.createAnyMixedContentType().withContent("Content Block Stuff Goes Here, STIX for example"))
                                    .withTimestampLabel(getTimestamp(null))
                                    .withMessage("Here's your data")
                    );
            
            String responseString = toXml(pollResponse);
            System.out.println(taxiiXml.marshalToString(pollResponse, true));
            
            return generateResponse(responseString, request);
        } catch(Exception ex) {
            ex.printStackTrace();
            return Response.status(Response.Status.INTERNAL_SERVER_ERROR).build();
        }
    }
    
    private Response generateResponse(String responseString, HttpServletRequest request) throws Exception {
        return Response.ok(responseString)
                    .header(HttpClient.HEADER_X_TAXII_PROTOCOL, getProtocol(request))
                    .header(HttpClient.HEADER_X_TAXII_CONTENT_TYPE, Versions.VID_TAXII_XML_11)
                    .header(HttpClient.HEADER_X_TAXII_SERVICES, Versions.VID_TAXII_SERVICES_11)
                    .build();
    }
    
    private String getProtocol(HttpServletRequest request) throws Exception {
        String scheme = new URI(request.getRequestURL().toString()).getScheme();
        if(scheme != null && scheme.equalsIgnoreCase("https")) {
            return Versions.VID_TAXII_HTTPS_10;
        } else {
            return Versions.VID_TAXII_HTTP_10;
        }
    }
    
    private void printHeaders(HttpServletRequest request) {
        System.out.println("--------------------------------------------");
        List headerNames = Collections.list(request.getHeaderNames());         
        for(String name : headerNames) {
            System.out.println(name + ": " + request.getHeader(name));
        }
    }

    private String toXml(Object discoveryResponse) throws Exception {
        final Marshaller m = taxiiXml.createMarshaller(false); 
        m.setProperty(Marshaller.JAXB_FRAGMENT, true); // Don't generate xml declaration.
        final StringWriter sw = new StringWriter();
        m.marshal(discoveryResponse, sw);
        return sw.toString();
    }

    private Object getRequestObject(String x) throws Exception {
        Unmarshaller um = taxiiXml.getJaxbContext().createUnmarshaller();
        return um.unmarshal(new StringReader(x));
    }

    private XMLGregorianCalendar getTimestamp(Date dte) throws Exception {
        if(dte == null) {
            dte = new Date();
        }
        GregorianCalendar gc = new GregorianCalendar();
        gc.setTime(dte);
        return DatatypeFactory.newInstance().newXMLGregorianCalendar(gc);
    }

Zenoss 3.2 Email Alerts

After finding a bunch of Zenos 4 documents on the ‘Trigger’ tab, I became frustrated that I couldn’t find how to send email alerts in Zenos 3.2.  The Zenoss Core UI is extremely confusing.

Alerts can be created per user and per group.  Go to Advanced->Users->click user->Alerting Rules

Here you can create filters for events and get emailed about them.

JVM proxy through a remote/reverse SSH tunnel

You may come across a scenario where you need to test your Java application from an alternate network, or you may not have sufficient network access where your code/application resides.  The JVM comes equipped with awesome proxy capabilities using SOCKs and a more specific HTTP proxy.  These can be set as environment properties when starting the JVM (http.proxyHost,http.proxyPort,socksProxyHost,socksProxyPort) or set programatically.  You can even configure a ProxySelector class for advanced proxy logic.  See Oracle’s Documentation for advanced usage.

Your http requests can go through SOCKs as well, but if the HTTPProxy is defined, then it will use the more specific proxy for the protocol it needs.

Here’s an example of the JVM options you would use to start your Java application to use proxies:

-Xmx2G -Dhttp.proxyHost=192.168.1.2 -Dhttp.proxyPort=80 -DsocksProxyHost=192.168.1.2 -DsocksProxyPort=1080

But, what if you don’t have direct access to the proxy server? You’ll need to use an SSH Tunnel. What if you can’t SSH out of your box? If you can SSH in, you can still create a remote/reverse tunnel. Here’s how that is done in Putty. Right click on the top of your active ssh putty window and chose ‘Change Settings’. Add the tunnels shown in the screenshots below to your connection. I noticed that sometimes ssh servers don’t have permissions to open up lower-numbered ports, so it took some trial and error before discovering that higher-numbered ports worked. The destination address of the tunnel is where the port will be forwarded to. The ‘Dynamic’ setting turns the tunnel into a SOCKs proxy itself and may be good if you don’t have a SOCKs server available, but I never got this working, but could have been because I was trying lower port numbers.

Putty1

Putty2

After you add those ports and apply the settings to your putty ssh session, your development machine now has two listening ports, one for http proxy, one for SOCKs proxy. Then you point your JVM at the localhost ports rather than the remote proxy ports.

-Xmx2G -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8555 -DsocksProxyHost=127.0.0.1 -DsocksProxyPort=8666

The SOCKs proxy in the JVM will even tell MySQL database connections to use the proxy. If you want a specific Apache HttpClient connection to use the proxy, you can configure it like this:

DefaultHttpClient client = new DefaultHttpClient();
HttpHost proxy = new HttpHost(“127.0.0.1”, 8555, “http”);
client.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);

or

client.getParams().setParameter(“socksProxyHost”,”127.0.0.1″);
client.getParams().setParameter(“socksProxyPort”, 8666);

If you are using Java’s URL class to make a request you can set a proxy for it:

Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(“127.0.0.1”, 8555));
HttpURLConnection connection = (HttpURLConnection) new
URL(myUrlString).openConnection(proxy);

If you are developing on a Windows box and need to SSH into that box, a good SSH Server that allows you to open up tunnels is COPSSH.

 

Java CAPTCHA w/ JCaptcha

When looking for a good Java CAPTCHA library, many people just suggested using ReCAPTCHA from Google.  A few online posts discouraged using ReCAPTCHA because it has shown swear words at times and has the possibility of being down which would also bring your own site down.  I like my Java Web Apps to have as few dependencies as possible, so I looked for a self-contained solution.  JCaptcha had everything I needed and was easy to integrate.  It was nice to be able to customize what kind of CAPTCHA words to use and what they would look like.

First off, add the dependency with Maven or as a Jar:

<dependency>
     <groupId>com.octo.captcha</groupId>
     <artifactId>jcaptcha</artifactId>
     <version>1.0</version>
 </dependency>

Add a custom engine to make it look how you want:

public class CustomCaptchaEngine extends ListImageCaptchaEngine {

     protected void buildInitialFactories() {

        com.jhlabs.image.WaterFilter water = new com.jhlabs.image.WaterFilter();
        water.setAmplitude(2d);
        water.setAntialias(true);
        water.setPhase(20d);
        water.setWavelength(70d);

        ImageDeformation backDef = new ImageDeformationByFilters(new ImageFilter[]{});
        ImageDeformation textDef = new ImageDeformationByFilters(new ImageFilter[]{});
        ImageDeformation postDef = new ImageDeformationByFilters(new ImageFilter[]{water});

        com.octo.captcha.component.word.wordgenerator.WordGenerator dictionaryWords = new com.octo.captcha.component.word.wordgenerator.ComposeDictionaryWordGenerator(
        new com.octo.captcha.component.word.FileDictionary("toddlist"));

        TextPaster randomPaster = new DecoratedRandomTextPaster(new Integer(6), new Integer(7), new SingleColorGenerator(Color.black), new TextDecorator[]{
        new BaffleTextDecorator(new Integer(1), Color.white)});
        BackgroundGenerator back = new UniColorBackgroundGenerator(new Integer(200), new Integer(75), Color.white);

        FontGenerator shearedFont = new RandomFontGenerator(new Integer(30), new Integer(35));

        com.octo.captcha.component.image.wordtoimage.WordToImage word2image;
        word2image = new DeformedComposedWordToImage(shearedFont, back, randomPaster, backDef, textDef, postDef);

        this.addFactory(new com.octo.captcha.image.gimpy.GimpyFactory(dictionaryWords, word2image));
    }
}

Then add an endpoint to serve up a BufferedImage.  I used Jersey for this:

public static ImageCaptchaService captchaService = new DefaultManageableImageCaptchaService(new FastHashMapCaptchaStore(), new CustomCaptchaEngine(), 180, 100000, 75000);

@Path("captcha.jpg")
@Produces("image/jpeg")
@GET
public Response lookupCategorization(@Context HttpServletRequest httpRequest) {

      BufferedImage bi = captchaService.getImageChallengeForID(httpRequest.getSession(true).getId());

 return Response.ok(bi)
      .header("Expires", 0)
      .header("Cache-Control", "no-store, no-cache, must-revalidate")
      .header("Pragma", "no-cache")
      .build();
 }

Add a validation method:

public static boolean validateResponse(HttpServletRequest request, String userCaptchaResponse) {
     if (request.getSession(false) == null) {
         return false;
     }
     boolean validResponse = false;
     try {
        validResponse = captchaService.validateResponseForID(request.getSession().getId(), userCaptchaResponse);
     } catch (CaptchaServiceException e) {}
     return validResponse;
}

Add a validation endpoint:

@Path("validate")
@Produces(MediaType.APPLICATION_JSON)
@POST
public Response validate(@Context HttpServletRequest httpRequest, @FormParam("captcha") String captcha) {

    if(validateResponse(httpRequest, captcha)) {
        ...
    } else {
        ...
    }
}

Add HTML form fields for CAPTCHA:

  <img class="captchaimage" src="captcha.jpg" />
  <input type="text" class="captchaText" autocomplete="off" name="captcha" /> 
  <a href="javascript: void(0);" onClick='loadNewCaptchaImage()'>Reload</a>

Add jQuery JavaScript to load a new image for users:

loadNewCaptchaImage = function () {
      $("img.captchaimage").attr("src", "captcha.jpg" + "?" + (new Date()).getTime()); 
      $('input.captchaText').val('').focus();
}

You can always make the captcha more difficult by adding more font variations, text effects, and background noise. Here are some examples of what you can do to make it difficult for bots to crack the captcha: https://jcaptcha.atlassian.net/wiki/display/general/Samples+tests  Just don’t frustrate your users by making it near impossible for them to figure out what the CAPTCHA says.