Migration Tutorial: Files Migration from Drupal 7 to Drupal 8

As per Wikipedia Data Migration is the process of transferring data from one system to another while changing the storage, database or application. In reference to the ETL (Extract-Transform-Load) process, data migration always requires at least Extract and Load steps.
 Like Migrating Data from Drupal 7 database to Drupal 8 database. We can perform Migration on file systems.
Migration can also happen to same (Drupal 7 to Drupal 7) or in between two other systems.

Where Upgrade is the process of replacing your existing software with a newer version of the same product. Like Upgrading your site from Drupal 7 to Drupal 8 version.
 
Here We'll focus on Migration of NODE or Content Type. Before we start reading about the NODE migration process we must be familiar with some basic things.
Please be sure that character sets (encoding) are set as same of Migration Source Data and Migration Destination Data like UTF-8.
Migration basically happens with three major steps i.e analyzing source data, prepare destination data structure and the process stays in between.
Say we are developing a module as Custom Migration module with Module key as "migrate_custom".
And yes we'll require to develop this for Drupal 8 :) by maintaining its standard.
migrate_custom will have some dependencies with other modules like Migrate (migrate), Migrate Plus (migrate_plus), Migrate Tools (migrate_tools), Migrate Drupal ...


Lets start with Files Migration with Code Example:
Our aim is to migrate all records from d7.file_managed to d8.file_managed through a command like
$ drush migrate-import custom_file --limit="500 items"
Ad per Drupal 8 standard we need a yml file to define everything and a php file to process that yml.
Here is there file/ folder structure
migrate_custom\src\Plugin\migrate\source\File.php
migrate_custom\config\install\migrate_plus.migration.custom_file.yml

In .yml file we define four major parts. These are id, source, process and destination. Id is to identify this Migration process. Source is defined with the data source. Process define basically the mapping of fields in between destination and source.

id: custom_file
source:
  plugin: custom_file
process:
  fid: fid
  filename: filename
  uri: uri
  uid:
    -
      plugin: default_value
      default_value: 1
destination:
  plugin: entity:file
status: true
migration_group: custom
dependencies:
  module:
    - file
label: Migrate D7 Files
migration_dependencies: {}

Inside File.php class File extends SqlBase { } with four basic public functions these are
1. query() to fetch data from the source.
2. fields() to defined needed fields.
3. getIds() to get and return fid of File.
4. prepareRow() is basically to set source Property. Here we are settings uri.

A working copy of this Migration is available at Github.

Drupal 8: Show content based on website visitor's country or content localization/ personalization.

Sometimes we need localization of contents i.e if a website visitor is browsing site from USA will be shown different content then someone is browsing from India or Germany.
So our aim is to show Content based on Visitor's Country, Geo IP, Geo Location.

To do so we'll extend the required content type with a filed say Country. While creating a new content there it will appear a Select Box to choose country as an Option. If this content is set to a country then it'll be available for that country only otherwise it'll be available for all countries.

Here is the technical guidance towards the solution.
Drupal Modules required

• Country
• Composer Manager
•  Address
• GeoIP

Regarding enabling those modules within Drupal please refer my another blog as "Drupal 8, Modules, Composer JSON, Platform.sh".
For example i've created one content type as "Country Based Content" with fields Title, Body, Country.
Where Country is field type Country.
Also created some contents of the above content type.

Created a view as "country" to show this content type in the Front-end of the website through view Module.
Its very easy but still i am attaching a screen-shot for beginners.

Usage of GeoIP Module :
GeoIP Module tells Drupal about the Country location of the visitor of the site based on Visitor’s
IP Address.
To explain it more technically,
GeoIP Module register a service as “geoip.geolocation” via its Constructor by following Drupal
Service registration standard.
We need to trace the website visitor’s IP v4 address through our PHP code and have to pass
this IP Address to GeoIP Module.
We use this code to trace the IP Address.
$visitorIp = \Drupal::request()->getClientIp();
As per my previous state we need to call the service:
\Drupal::service('geoip.geolocation')->geolocate('14.140.162.242');
So here is the output below:
CommerceGuys\Intl\Country\Country Object
(
[countryCode:protected] => IN
[name:protected] => India
[threeLetterCode:protected] => IND
[numericCode:protected] => 356
[currencyCode:protected] => INR
[locale:protected] => en
)
Now we’ll access property of Country Object from its Data Model.
We need to get Country Code value from countryCode” property. So this code below will return
“IN” as my country code.
\Drupal::service('geoip.geolocation')->geolocate($visitorIp)->getCountryCode();

localization of Content for Website Visitors:
As i am browsing this site from from India so it’ll display only contents belong to India.
Let’s explain a little bit in coding level.
To alter the view we’ve used hook_views_query_alter()
Where we’ve traced the view id to alter.
function geoip_content_views_query_alter(Drupal\views\ViewExecutable $view,
Drupal\views\Plugin\views\query\Sql $query) {
switch ($view->id()) {
case 'country':
$data = 'IN';
$query->where[2]['conditions'][1]['value'] = [$data];
break;
}
}

Drupal 8, Modules, Composer JSON, Platform.sh

In Drupal CMS we can integrate Modules in different ways like uploading module via module manager/ extent, manually copy pasting module folders, via Drush command, Drupal consol and Composer.

Most preferable ways are either via Drush or by Composer. Now we are sometimes confused what is best way to do that actually. My personal opinion is whatever procedure/ technique u've applied must reach the  requirement properly. I also prefer by Dursh or by Composer.
Now some modules has dependency on third party libraries. Drush is unable to load those vendor libraries.
Better explain through an example.
Lets consider about Address Module. Address has third party vendor/ library dependency and if we execute command like

$drush cr --continue-with-error/ --no-halt-on-error
$drush en Address -y

It'll successfully download the Address module but will throw error about Vendor dependency i.e. Address Module will not be enabled.

So, drush fails to enable Address module. Here we need Composer extensively. Composer checks the Module and its Vendors defined in composer.json file. So we need to execute commands like this -

$composer config repositories.drupal composer https://packages.drupal.org/8
$composer require "drupal/address ~1.0"

Similarly to add geoip Module we can execute this command.

 $composer config repositories.drupal composer https://packages.drupal.org/8
$composer require "drupal/geoip 2.x-dev"

Now what should we do when we are on Platform.sh ?
To be continued ...

Update Day - Date Time based on systems local time-zone of Website Visitor.

The headline is not quiet clear itself. Definitely it needs some explanation what it actually want to tell. Going through with an example will be a good approach for me to clarify.

Say, there is a WebInar as "What CSS3 can do?" is going to happen at January 16, 2017 10.00 PM
And  considering this WebInar is going to happen on IST time i.e. GMT+5:30 or UTC+5:30

Lets say HTML tag structure is like below
<ul class="webinar-info">
 <li><i class="fa fa-calendar"></i> January 16, 2017</li>
 <li><i class="fa fa-clock-o"></i> 10:00 PM</li>
</ul>

Now our JavaScript code will follow these flowchart to convert this IST to another time zone based on website visitors local system. i.e. If some is browsing this site from EST zone this IST will appear to the visitor on EST time.

• Read Date Time from the html tag and convert this to time-stamp.
• Converting this IST time-stamp to UTC-GMT time-stamp.
• Get time offset of the Visitor's timezone.
• Calculate the Visitor's local time-stamp based on GMT time-stamp and time offset.
• Now, generate local date, time, year based on the calculated local time-stamp.
• Display in the required format by over-writing it.

Here is a sample code that does this same thing above.

var pageInitialized = false;
Drupal.behaviors.webinar_upcomingevent = {
    attach: function(context) {
    if(pageInitialized) return;
    pageInitialized = true;
        ;(function($){
            //return ;
            if($("ul.webinar-info li:first").text().trim().length > 1 ){
                var wInarDateText    = $("ul.webinar-info li:first").text().trim();
                var wInarDate     = wInarDateText.split(" ");    //alert(wInarDate[1].replace(/^,|,$/g,''));

                var wInarTimeText    = $("ul.webinar-info li:last").text().trim();   
                var hours = Number(wInarTimeText.match(/^(\d+)/)[1]);
                var minutes = Number(wInarTimeText.match(/:(\d+)/)[1]);
                var AMPM = wInarTimeText.match(/\s(.*)$/)[1];
                var AMPM = AMPM.toUpperCase();
                if(AMPM == "PM" && hours<12) {
                     hours = hours+12;
                }
                if(AMPM == "AM" && hours==12) {
                     hours = hours-12;
                }
                var sHours = hours.toString();
                var sMinutes = minutes.toString();
                if(hours<10) sHours = "0" + sHours;
                if(minutes<10) sMinutes = "0" + sMinutes;

                var monthNumber = ["january", "february", "march", "april", "may", "june", "july", "august", "september", "october", "november", "december"].indexOf(wInarDate[0].toLowerCase());
                //var monthNumber = monthNumber + 1;
                var wInarIST    = new Date(wInarDate[2],monthNumber,wInarDate[1].replace(/^,|,$/g,''),sHours,sMinutes).getTime();
                // Perfect timestamp has been calculated "wInarIST"
                //alert(new Date(wInarIST).toUTCString());
                //Considering TimeZone = IST = +5Hrs30mins = +330Mins to milliseconds
                var utcGmtTStamp    = wInarIST - 330*60*1000;
                var offset             = new Date().getTimezoneOffset()*60*1000;
                var localTimeStamp    = utcGmtTStamp - offset;
                //Local timestamp calculation is perfect
                //alert('ist='+wInarIST+'utcGmtTStamp='+utcGmtTStamp+'offset='+offset+'localTimeStamp='+localTimeStamp);
                var localDate    = new Date(localTimeStamp);
                var yearL    = localDate.getFullYear();
                var monthL   = localDate.getMonth();
                var dayL     = localDate.getDate();
                var hourL    = localDate.getHours();
                var minuteL  = localDate.getMinutes();
                //var secondsL = localDate.getSeconds(); 
                var amPmLocal    = hourL >= 12 ? 'PM' : 'AM';
                hourL = hourL % 12;
                hourL = hourL ? hourL : 12; // the hour '0' should be '12'
                minuteL = minuteL < 10 ? '0'+minuteL : minuteL;

                var monthArray = new Array();
                monthArray[0] = "January";
                monthArray[1] = "February";
                monthArray[2] = "March";
                monthArray[3] = "April";
                monthArray[4] = "May";
                monthArray[5] = "June";
                monthArray[6] = "July";
                monthArray[7] = "August";
                monthArray[8] = "September";
                monthArray[9] = "October";
                monthArray[10] = "November";
                monthArray[11] = "December";
                //alert('monthNumber='+monthNumber+'year='+year+'month='+month+'day='+day+'hour='+hour+'minute='+minute);
                $("ul.webinar-info li:first").html('<i class="fa fa-calendar"></i> '+monthArray[monthL]+' '+dayL+', '+yearL);
                $("ul.webinar-info li:last").html('<i class="fa fa-clock-o"></i> '+hourL+':'+minuteL+' '+amPmLocal);
            }
        })(jQuery);
    }
};

This code is based on Drupal 8 behaviour along with some jQuery dependency.
This code also checks whether this it's executing more then once.

Drupal 8: Validate a file while uploading via AJAX.

In Drupal 8 file uploading is done beautifully with its builtin AJAX support. Now visitor can upload many dump files into your server. This file uploading also gets entry into database with status 0 ;) Or may be you want to allow validated CSV file to be in server for further process.
Say, User is uploading test.csv file. You are checking the content of CSV file is as per required format and acceptable.
To do so you need to take help of hook function in your .module file. Like
function HOOK_file_validate(FileInterface $file) {
}
Here's the complete code :
use Drupal\file\FileInterface;

/**
 * hook function to check CSV file while uploading
 * $dataRow[0]->Name->string,  $dataRow[1]->Mobile->number
 * @param $file
 * @return array
 *
 */
function osu_studentdataupload_file_validate(FileInterface $file) {
  // Get Type of The Node so that this function validate only the specific.
  $node_type = getNodetypeFromUrl();
  if ($node_type == 'myNodeType') {
      //$file->id;
      // Getting Temp path :: we still don't have fully access to the file
      $filePath = $file->getFileUri();
      //$file->getFilename();
      $errors = [];
      foreach (getCsvFileRow($filePath) as $dataRow) {
        //if (!empty($dataRow[0]) && (1 === preg_match('~[0-9]~', $dataRow[0]))) {
        if (1 === preg_match('~[0-9]~', $dataRow[0])) {
          $errors[] = '"' . $dataRow[0] . '" - has number(s).';
        }
        if ((!empty($dataRow[1])) && (0 === preg_match('~[0-9]~', $dataRow[1]))) {
          $errors[] = '"' . $dataRow[1] . '" - has string.';
        }  
      }
      return $errors;
      }
  }
}

/**
 * Get NodeType from anywhere inside Drupal from Request url.
 *
 * @return string
 * The current node_type.
 *
 */
function getNodetypeFromUrl() {
  // While Creating
  $nodeTypeObj = \Drupal::request()->attributes->all()['node_type'];
  if (is_object($nodeTypeObj)) {
    $node_type = accessProtected($nodeTypeObj, 'type');
  }
  // While Editing
  $nodeObj = \Drupal::request()->attributes->all()['node'];
  if (is_object($nodeObj)) {
    $nodeObjToArray = (array) $nodeObj;
    $prefix = chr(0) . '*' . chr(0);
    $node_type = $nodeObjToArray[$prefix . 'values']['type']['x-default'];
  }
  return $node_type;
}

/**
 * Access protected property of Object by using ReflectionClass.
 *
 * @param Object
 * An Object containing public/restricted property.
 * @param string
 * The name of the property need in return.
 *
 * @return
 * Value of the required property.
 *
 */
function accessProtected($obj, $prop) {
  $reflection = new ReflectionClass($obj);
  $property = $reflection->getProperty($prop);
  $property->setAccessible(true);
  return $property->getValue($obj);
}

/**
 * CSV File format handler.
 *
 * @param file path
 * A CSV file path to read contents.
 *
 * @return array
 * Array of data of the current line.
 *
 */
function getCsvFileRow($bigDataFile) {
  $handler = fopen($bigDataFile, 'r');
  if ($handler === false) {
    throw new Exception();
  }
  while (feof($handler) === false) {
    yield fgetcsv($handler);
  }
  fclose($handler);
}

Understanding Camunda BPM at Basic Level and deploying a BPMN workflow.

Here i am trying to share some understanding for beginners by writing down a technical blog regarding
Camunda BPM
It's specially for my reference to recall and understand it at a glance whenever needed and may be this is helpful to someone else as well ;)

Camunda BPM Basics:
Camunda BPM is a JAVA based open-source framework for Business Process Management.
Camunda implements 3 different standards in the BPM and these are

• BPMN 2.0 - Business Process Model Notation.
• CMMN 1.1 - Case Management Model and Notation.
• DMN 1.1 - Decision Model and Notation.

I am not elaborating this blog by describing about Camunda Modeller. Considering that Reader is already aware of that or please refer https://docs.camunda.org/manual/.
The Camunda BPM web applications are based on a RESTful architecture.
JAX-RS based Rest API, AngularJS, RequireJS, jQuery, Twitter Bootstrap are the used frameworks for it.
Camunda BPM consists with Cockpit, Tasklist, Admin section.
Cockpit -  Monitoring and operations that allows you to search for process instances, inspect their state and repair broken instances.
Tasklist - A web application for human workflow management and user tasks that allows process participants to inspect their workflow tasks and navigate to task forms in order to work on the tasks and provide data input
Admin - It allows you to manage users, groups and authorizations.

I am not going to discuss more detail about its architecture, extensions, features. Camunda BPM also helps to generates various reports to its deployed workflows.

Here i'll only discuss with BPMN diagram in brief and its simple deployment with example.

BPMN Basics:
BPMN implementation done with several components.
Like Participants (Pool, Lane), Subprocesses (Subprocess, Transaction, Event Subprocess), Tasks (Service, User, Script), Gateways (XOR, OR, AND), Events (Start, End, Termination, Message)

Designing a BPMN Diagram:

By using Camunda Modeler tool we design the required BPMN workflow diagram.
This Modeling tool is platform independent.
We also set the form variables for gateways while modelling.

Deployment Strategy:
Designing BPMN workflow Model and deploying it inside Camunda Engine is a lengthy process along with some prerequisite environment settings like Java JDK 1.7+ and Apache Maven.

Deployment steps are listed below:

1. Installation of Camunda BPM Platform and Camunda Modeler. 
• Download an Apache Tomcat bassed standalone distribution of the Camunda BPM platform.
• Run Camunda by execute the script named start-camunda.bat(Windows) or start-camunda.sh (Unix).
• In Browser Camunda will become available in this url http://localhost:8080/camunda-welcome/index.html
• Download the latest Camunda Modeler release.
• Modeler can then be started by running the executable named camunda-modeler.
2. Set up an Apache Maven-based process application inside Eclipse.
• Add Camunda Maven Dependencies to Eclipse JAVA IDE by adding some of its plugins.
3. Model a diagram through Camunda Modeller. (https://docs.camunda.org/get-started/bpmn20/model/)
• Create a new BPMN Diagram.
• While creating set the task names properly for better understanding and variables for various GATEs.
4. Generating a Project .war file and Exporting through Apache-Maven. 
• Add a Process Application Class to the Project. 
• Add a META-INF/processes.xml Deployment Descriptor file.
5. Deploying and Checking a .war file inside Camunda BPM.
• Need to move .war file from Maven project to the $CAMUNDA_HOME/server/apache-tomcat/webapps folder.
• Test the BPMN 2.0 Process with Camunda Tasklist and Camunda Cockpit.
• Now use Cockpit to check if the process is successfully deployed. Go to http://localhost:8080/camunda/app/cockpit
• Start a Process Instance.
• In Camunda Tasklist ( http://localhost:8080/camunda/app/tasklist ) click on the Start process button to start a process instance.
• If required Configure Process Start Authorizations.
• To Work on the Task go to Tasklist ( http://localhost:8080/camunda/app/tasklist ) and log back in with the user credentials.

Some More Information:
PHP SDK is available here http://camunda.github.io/camunda-bpm-php-sdk/ and
**If there is any change in BPMN Model/ Workflow we need to Re-Deploy inside Camunda by re-generating a new “.war” file :)

Screenshot of a Deployment of BPMN Model in Eclipse.