Showing Remote Image using ImageView in BB10 Cascades

While working on my BB10 App update, I required to show Remote Image using ImageView BB10 cascades API. ImageView or Image component by default does not support loading image from URL on internet. I implemented a custom component which serve the purpose and also easily integrated with ImageView.

Following is implementation for the same, I hope it will be useful to someone.

Let's start by showing how my custom component works. Following code shows how to use ImageDownloader custom component along with ImageView to display Grid of images from internet. Data Model contains two kind of URL, url for low resolution image and url for high resolution image.

import my.library 1.0

ListView {
  id: listView
  
  layout: GridListLayout {}
  dataModel: model;
  
  listItemComponents: [
      ListItemComponent {                        
          ImageView {
              id: imgView
              imageSource: "default_image.jpg"  
                                        
              attachedObjects: [
                  ImageDownloader {
                      id: imageLoader
                      url: ListItemData.mediumImage
                      onImageChanged: {                 
                          if (imageLoader.isVaid) {
                              imgView.image = imageLoader.image
                          }
                      }
                  }
              ]
          }
      }
  ]
}

ImageDownloader component is available in QML because we imported "my.library". We can make any C++ code available to QML by registering it to QML System, folloiwing snippet shows how we can register C++ component to QML system.

#include "ImageDownloader.h"

int main(int argc, char **argv) 
{
 Application app(argc, argv);

 qmlRegisterType<ImageDownloader>("my.library",1, 0, "ImageDownloader");

 QmlDocument *qml = QmlDocument::create("asset:///main.qml").parent(&app);

 AbstractPane *root = qml->createRootObject();
 app.setScene(root);

 return Application::exec();
}

Now that custom component is ready to be used with QML, let's see how its implemented. Below if header file for ImageDownloader class. We are defining few properties like url, image and isValid. By setting "url" we can initiate download of image, when "image" download is finished downloaded image can be accessed by using "image" property. We can check if image is valid or not by checking "isValid" property. we are also defining few signal like "urlChanged" and "imageChanged", which are emited when url is changed or image is downloaded. And we are using QNetworkAccessManager to download image from internet.

#ifndef IMAGEDOWNLOADER_H_
#define IMAGEDOWNLOADER_H_

#include 
#include 

class QNetworkAccessManager;

class ImageDownloader: public QObject
{
    Q_OBJECT
    Q_PROPERTY(QVariant image READ image NOTIFY imageChanged)
    Q_PROPERTY(QString url READ url WRITE setUrl NOTIFY urlChanged)
    Q_PROPERTY(bool isVaid READ isValid CONSTANT);
public:
    ImageDownloader( QObject* parent = 0);
    virtual ~ImageDownloader();

signals:
    void urlChanged();
    void imageChanged();

private slots:
    QString url() const;
    void setUrl( const QString& url);

    QVariant image() const;
    bool isValid() const;

    void startDownload();
    void onReplyFinished();

private:
    QNetworkAccessManager mNetManager;
    QString mImageUrl;
    bb::cascades::Image mImage;
    bool mIsValid;
};

#endif /* IMAGEDOWNLOADER_H_ */

Implementation is also quite simple, let's see how its implemented. On URL change, we are initiating the image download using QNetworkAccessManager. When download is finished, we are reading image data in to buffer and creating Image using BB10 Image API.

#include "ImageDownloader.h"

#include <QNetworkAccessManager>
#include <QNetworkRequest>
#include <QNetworkReply>

ImageDownloader::ImageDownloader(QObject* parent):
    QObject(parent), mIsValid(false) {}

ImageDownloader::~ImageDownloader() {}

QString ImageDownloader::url() const {
    return mImageUrl;
}

void ImageDownloader::setUrl( const QString& url)
{
    if(url != mImageUrl) {
        mImageUrl = url;
        mIsValid = false;
        mImage = bb::cascades::Image();
        emit urlChanged();
        emit imageChanged();
        startDownload();
    }
}

QVariant ImageDownloader::image() const {
    return QVariant::fromValue(mImage);
}

bool ImageDownloader::isValid() const {
    return mIsValid;
}

void ImageDownloader::startDownload() {
    QNetworkRequest request(mImageUrl);
    QNetworkReply* reply = mNetManager.get(request);
    connect(reply, SIGNAL(finished()), this, SLOT(onReplyFinished()));
}

void ImageDownloader::onReplyFinished() {
    QNetworkReply* reply = qobject_cast(sender());
    QString response;
    if (reply) {
        if (reply->error() == QNetworkReply::NoError) {
            const int available = reply->bytesAvailable();
            if (available > 0) {
                const QByteArray data(reply->readAll());
                mImage = bb::cascades::Image(data);
                mIsValid = true;
                emit imageChanged();
            }
        }
        reply->deleteLater();
    }
}

That's all we have to do to do display remote image in BB10, hope you liked it.

Using QML Camera and passing image to C++ code

I tried to compile one my application with Qt5. Application was using QML camera and sharing image to C++ code for further processing.

Following is sample code, it works with Qt5 and Qt Multimedia 5.

Lets start with ImageProcessor class, the C++ class which is called from QML to further image processing.

Following is header file for ImageProcessor class, it declares processImage() slot which can be invoked from QML code.

#ifndef IMAGEPROCESSOR_H
#define IMAGEPROCESSOR_H

#include <QObject>

class ImageProcessor : public QObject
{
    Q_OBJECT
public:
    explicit ImageProcessor(QObject *parent = 0);
 
public slots:
    void processImage( const QString& image);   
};
#endif // IMAGEPROCESSOR_H

Following is cpp file for ImageProcessor class. processImage() function retrieves Image from camera image provider. Once we have valid image, we can process it further.

#include "imageprocessor.h"
#include <QtQml/QmlEngine>
#include <QtQml/QmlContext>
#include <QtQuick/QQuickImageProvider>
#include <QDebug>

ImageProcessor::ImageProcessor(QObject *parent)
    : QObject(parent)
{}

void ImageProcessor::processImage( const QString& path)
{
    QUrl imageUrl(path);
    QQmlEngine* engine = QQmlEngine::contextForObject(this)->engine();
    QQmlImageProviderBase* imageProviderBase = engine->imageProvider(
     imageUrl.host());
    QQuickImageProvider* imageProvider = static_cast<QQuickImageProvider>
     (imageProviderBase);
    
    QSize imageSize;
    QString imageId = imageUrl.path().remove(0,1);
    QImage image = imageProvider->requestImage(imageId, &imageSize, imageSize);
    if( !image.isNull()) {
        //process image
    }
}

Now we need to register ImageProcessor class with QML. so that we can use it from QML code. This can be done by using qmlRegisterType global function.

#include  <QtGui/QGuiApplication>
#include  <QQmlEngine>
#include  <QQmlComponent>
#include  <QtQuick/QQuickView>

#include "imageprocessor.h"

int main(int argc, char *argv[])
{
    qmlRegisterType<ImageProcessor>("ImageProcessor", 1, 0, "ImageProcessor");

    QGuiApplication app(argc, argv);

    QQuickView view;

    QObject::connect(view.engine(),SIGNAL(quit()),&app,SLOT(quit()));    
    view.setSource(QUrl::fromLocalFile("qml/main.qml"));
    view.show();

    return app.exec();
}

That's all from C++ side, QML code is event easier. Following how you can use ImageProcess class from QML code.

import QtQuick 2.0
import QtMultimedia 5.0
import ImageProcessor 1.0

Rectangle {
    width: 360
    height: 360

    //shows live preview from camera
    VideoOutput {
        source: camera
        anchors.fill: parent
        focus : visible
    }

    //shows captured image
    Image {
        id: photoPreview
        anchors.fill: parent
        fillMode: Image.PreserveAspectFit
    }

    Camera {
        id: camera
        imageProcessing.whiteBalanceMode: CameraImageProcessing.WhiteBalanceFlash
        captureMode: Camera.CaptureStillImage

        exposure {
            exposureCompensation: -1.0
            exposureMode: Camera.ExposurePortrait
        }

        flash.mode: Camera.FlashRedEyeReduction

        imageCapture {
            onImageCaptured: {
                photoPreview.source = preview
                imageProcessor.processImage(preview);
            }
        }
    }

    MouseArea{
        anchors.fill: parent
        onClicked: {
            camera.imageCapture.capture();
        }
    }

    //image processor for further image processing
    ImageProcessor{
        id: imageProcessor
    }
}

Tracking color in image and detecting gesture in Qt

In this blog post I have written how we can access the individual frame from QCamera, In this blog post I will show how to use those frame to track some particular colored object and detecting gesture from motion of that object.

Following is demo of my sample application running on N9.

Tracking colored object

I don't know the actual algorithm for detecting color in image but i created simple algorithm that will detects some predefined color in image. Please note that if image has multiple object with the color which we are tracking it will return rectangle which cover all objects, not individual rectangle of each object.

As I am not interested in details of captured image,  just interested in checking if image has defined color object or not, I reduces size of image to half, so i have to process small number of pixel to detect color.

Then to detect color in image, I convert image capture from camera from RGB color spec to HSV color spec, as its quite easy to process HSV color spec to detect color.

After image is converted to HSV color spec, I converted image to black and white image, black portion will be detected object and rest of things will be in white. After getting this image I just need to scan image to find area of black portion of image.

So now I have coordinate of colored object which we are detecting.

Following code implements the above logic to detect the red colored object, in code I combined process of converting image to black and white and detect the black portion of image.

QRect ColorMotionDetector::detectColor( const QImage& origImage)
{
    //reduce size of image
    QImage image(origImage);
    image = image.scaled(QSize(320,240));

    emit originalImage(image);

    //rectanlge of detected colored object
    int maxX = -1;
    int minX = 99999;
    int maxY = -1;
    int minY =  99999;

    int width = image.width();
    int height = image.height();
    bool detected = false;

    //black and white image
    QImage converted(image.size(),image.format());

    for (int y = 0; y< height; ++y ) {
        for( int x = 0; x < width; ++x ) {
            //convert individual pixel to HSV from RGB
            QRgb pixel = image.pixel(x,y);
            QColor color(pixel);
            color = color.toHsv();

            
            //default whitel color for other object
            QRgb newPixel = qRgb(255, 255, 255);
            
            //detecting red color
            if( color.hue() >= 0 && color.hue() <= 22
                    && color.saturation() <= 255 && color.saturation() >= 240
                    && color.value() <= 255 && color.value() >= 100 ) {

                detected = true;

                if( x > maxX ) {
                    maxX = x;
                } else if( x < minX )  {
                    minX = x;
                }

                if( y > maxY ) {
                    maxY = y;
                } else if( x < minY )  {
                    minY = y;
                }
                
                //black color for detected object
                newPixel = qRgb(0, 0, 0);
            } 
            converted.setPixel(x,y,newPixel);
        }
    }

    QRect rect;
    if( detected) {
        rect = QRect(minX,minY, maxX - minX, maxY-minY );

        //drawing red rectangle around detected object
        QPainter painter( &converted );
        painter.setPen(QPen(Qt::red));
        painter.drawRect(rect);
        painter.end();
    }
    emit processedImage(converted);

    return rect;
}

Detecting swipe gesture

When we detect the position of object using above color detection code, we can use that position to detect if position tracked from individual image create some kind of gesture.

I will show how to use captured position to detect horizontal swipe gesture, we can easily extend it to detect vertical swipe or diagonal swipe.

I used following logic to detect swipe gesture,

> As color detection code returns position of tracked object, We compare this new position with its old position.
> If there is any progress in motion of object, we add difference of x coordinate to total progress made. In case of no progress, we discard whole gesture and reset variable that keep track of motion.
> While doing so if we detect certain amount of movement in particular direction, we decide if gesture was left swipe or right swipe using difference in position of object and reset the variables.
Following code implement above logic.

Gesture ColorMotionDetector::detectGesture(QRect rect) {

    //not valid rectangle, mean no object detected
    if( !rect.isValid()) {
        mLastRect = QRect();
        mXDist = 0;
        return Invalid;
    }

    //there is no previous cordinate, store rect
    if( !mLastRect.isValid() ) {
        mLastRect = rect;
        mXDist= 0;
        return Invalid;
    }

    Gesture gesture = Invalid;
    int x = rect.x();
    int lastX = mLastRect.x();
    int diff = lastX - x;

    mLastRect = rect;
    //check if there is certain amount of movement
    if( qAbs( diff ) > 10 ) {
        //there is movement in x direction, store amout of movement in total movement
        mXDist += diff;
       
        //x motion match to amount required for perticular gesture
        //check if motion of let to right or right to left
        if( mXDist >  150 ) {
            qDebug() << "Right horizontal swipe detected..." << mXDist;
            mXDist = 0;
            gesture = SwipeRight;
        } else if ( mXDist < -150 ) {
            qDebug() << "Left horizontal swipe detected..." << mXDist;
            mXDist = 0;
            gesture = SwipeLeft;
        }
    } else {
        //discard the gesture
        mXDist = 0;
        mLastRect = QRect();
    }
    return gesture;
}

Putting all together

Now we have code that detect colored object and code that detect gesture. Following code shows how those function are used together.

//detection motion from captured image from camera
void ColorMotionDetector::detectMotion( const QImage& image) {

    QRect rect = detectColor( image);
    Gesture gesture = detectGesture( rect );

    if( gesture != Invalid ) {
        emit gestureDetected( gesture );
    }
}

Following is vary simple gesture handler, which just print handled gesture.

void MyWidget::gestureDetected( Gesture gesture) {

    if( gesture ==  SwipeLeft) {
        mSwipeLabel->setText("Left swipe");
    } else if( gesture == SwipeRight) {
        mSwipeLabel->setText("Right swipe");
    }
}