1. Home
  2. Internet of Things
  3. Internet of Things Overview

Internet of Things Overview

Azure IoT Launchpad

The Big Picture for IoT Solutions

 

 

 

 

Things generate DATA which is analyzed in motion and at rest to gain INSIGHT from which intelligent ACTION is taken.

 

 


Intelligence

 

in the Cloud

 

and

 

on the Edge

 

 

 

Security

Across

the

Entire

Solution

 

Capabilities

for

SaaS
&
PaaS

Solutions

 

 

Helpful IoT Links

  • Azure IoT – main website for Azure IoT.
  • Azure IoT Fundamentals – a technical QuickStart for understanding the fundamentals of Azure IoT.
  • Azure IoT Security – best practices for IoT security.
  • Solution accelerators – open source templates to create customizable PaaS solutions for common IoT scenarios.
  • IoT Blog – Microsoft’s public blog about IoT.
  • Product Documentation – documentation portal for the main tools and services supporting IoT solutions.
  • Azure IoT Reference Architecture – a document detailing best practices for architecting IoT solutions.
  • Azure IoT School – online courses and labs to learn how to develop Azure IoT solutions.
  • Azure IoT Developer Center – a site for IoT developers with links for documentation, tutorials, and training.
  • Azure IoT Edge – main website for Azure IoT Edge.
  • Azure IoT Hub on Azure Friday – Olivier Bloch joins Scott Hanselman to discuss Azure IoT and how it is more than just about connecting IoT devices and sending telemetry to the Cloud. They also talk about Azure IoT device topics such as twins, provisioning, and lifecycle management.
  • Azure IoT Edge at Build – Video of Microsoft Build Live 2018 discussing Azure IoT Edge: a breakthrough platform and service running cloud intelligence on any device.
  • Azure IoT SDKs – The benefits of using Azure IoT SDKs for your devices and the best practices to implement if you don’t.
  • Azure Data Storage Options – Typically IoT ends up generating Big Data. Evaluate your big data storage technology in Azure.
  • Azure Function Recipes – Functions recipes are a collection of community-maintained tips and tricks for working with Azure Functions.

IoT Architecture Overview

As described in the Azure IoT Reference Architecture document, best practices for a scalable and flexible IoT solution include an architecture with the following attributes.

 

  • cloud native
  • microservice and serverless based
  • with subsystems having discrete services that are independently

o    deployable

o    scalable horizontally via

  • an orchestrator (Azure Managed Kubernetes or Service Fabric)
  • or PaaS services (Azure App Services)

o    monitorable

o    communicate over REST/HTTPS using JSON (unless high performance requires a binary protocol)

  • supports a hybrid cloud and edge compute strategy

 

The subsystems can be thought of in the following manner.

  • Core Subsystems (in green)
  • Optional Subsystems (in blue)
  • Cross-Cutting Needs (across the entire solution)

 

 

 

Technology Choices for Core Subsystems

IoT Devices and On-premise Edge Gateways

Devices and on-premise edge gateways securely register with the cloud and send and receive messages.

Cloud Gateways and Hubs

Cloud gateways and hubs securely accept data from devices and provide device management capabilities.  Use the following technologies for these purposes.

  • Azure IoT Hub Service – provides built-in secure connectivity, telemetry and event ingestion, and bi-directional communication and device management.  Key features include:  hyper scale, secure communications, built-in message routing, integration with other services, and high-availability SLA,  For an overview see What is IoT Hub?
  • Azure IoT Device Provisioning Service – provides assignment, configuration, and registration of devices at scale.
  • Azure IoT Hub SDKs – are used for securing device connectivity and sending telemetry data.

Stream Processors

Stream processors consume device data, place data into storage, and integrate with business processes.

Business Processors

Business processors execute actions based on device telemetry data from stream processing.

Storage (Warm and Cold)

Warm storage

A warm storage database stores device state for a pre-determined recent interval and may also store an easily accessible last known state per device. This data must be available in the database quickly (ideally within a matter of seconds from when the data is ingested into the cloud gateway from the device) and easily queried for simple scenarios such as visualizing current device sensor values or visualizing values over a recent timeframe.

  • Azure Cosmos DB – a secure, highly scalable (no limits on data storage or throughput), low latency NoSQL database. It is best for datasets that can benefit from flexible, schema-agnostic, automatic indexing, and rich query interfaces. Azure Cosmos DB has 5 API types and data models—SQL, MongoDB, Graph, Table and Cassandra, which provide the flexibility to choose a data model based on the data needs of the solution. Cosmos DB allows multi-region read and write and supports manual failover in addition to automatic failover. In addition, Cosmos DB allows the user to set a time-to-live (TTL) on their data, which makes expiring old data automatic.  For documentation on security of Cosmos DB see:  https://docs.microsoft.com/en-us/azure/cosmos-db/database-security
  • Azure Time Series Insights – an analytics, storage and visualization service for time-series data, providing capabilities including SQL-like filtering and aggregation, alleviating the need for user-defined functions. All data in Time Series Insights is stored in-memory and in SSDs, which ensures that the data is always ready for interactive analytics. For example, a typical aggregation over tens of millions of events returns on the order of milliseconds. It also provides visualizations such as overlays of different time series, dashboard comparisons, accessible tabular views, and heat maps. Time Series Insights provides a data explorer to visualize and query data as well as REST Query APIs. Further, it exposes a JavaScript controls library that make it simple to embed Time Series Insights-powered charts into custom applications. TSI is suited for solutions that need visualization services built in and do not need to report on data immediately (TSI has an approximate latency for querying data records of 30-60 seconds). TSI is well-suited for solutions that need to query aggregates over large sets of data, as TSI allows any number of users to conduct an unlimited number of queries for no extra cost. TSI has a maximum retention of 400 days and a maximum storage limit of 3 TB, so a solution using TSI will need to use a cold storage database (likely swapping data into TSI for querying as needed) as well if the customer needs a larger amount of storage or retention. TSI is our recommendation for time series data storage and analytics
  • Azure SQL Database – best for datasets that require relational storage and query capabilities. SQL Database also provides advanced features for data management, protection and security, and business continuity. Pricing is based on a combination of storage provisioned and Database Transaction Units or elastic Database Transaction Units provisioned. Manual scaling to increase storage space has no downtime. SQL DB also has built in replication and automatic region failover to ensure data is not lost in an outage.  However, there are limits on scale and throughput for write ingest so make sure your use case fits within these constraints.
  • Apache Cassandra (third party) – a linearly scalable, highly available NoSQL database and can span clusters across geographical regions. It uses the CQL query language, which is modeled after SQL. It offers authentication, encryption and firewall capabilities, as well as data replication. In addition, it performs well in write-heavy scenarios (it can achieve over 1 million writes per second), which makes it a good fit for an IoT solution that has a high level of telemetry ingestion. use for write intensive scenarios (can handle over 1 million writes per second.)

Cold storage

Instead of keeping all data in a warm data store with low latency, high throughput, and full query capabilities, data can be split into warm and cold storage paths. This can provide lower storage costs while still preserving historical data. A cold storage database holds data that is not needed as quickly and/or frequently as warm storage, but still may be necessary to access in the future for reporting, analysis, machine learning use, etc.

The best cold storage database for a solution is highly dependent on what purpose the database will serve. The two data storage solutions below are designed for high scale at a low price, but each has strengths for different scenarios. We recommend Azure Blob Storage for the general case, as it is cheaper than Azure Data Lake, especially in terms of write requests, is currently available in more regions, and has better disaster recovery. However, if the solution requires cold storage data analytics (with Hadoop, Azure Data Analytics, etc.), or requires querying with U-SQL, Data Lake is designed with that scenario in mind and may be the better choice.

  • Azure Blob Storage – a simple, inexpensive file storage database. Blobs can be used to store raw device data. Using page blobs instead of block or append blobs should be considered depending on frequency of write operations.  Azure blob storage has full security capabilities, local or geo-redundant storage options, and is available in all Azure regions. It is highly scalable—the maximum storage limit is 500 TB and the maximum request rate per account is 20,000 requests per second.
  • Azure Data Lake – a distributed data store that can persist large amounts of relational and nonrelational data without transformation or schema definition. It is a good choice for a storage database if big data analytics and/or unlimited storage are required. It is slightly more expensive than Azure Blob Storage (specifically in terms of write operations), but it is optimized for big data analytics workloads. The database can be accessed from Hadoop via WebHDFS-compatible REST APIs or using the U-SQL language. It has locally redundant storage and is available in some US Azure regions as well as North Europe.
  • Notes:

User Interface

User interfaces in IoT applications visualize telemetry data and facilitate device management.

  • Azure App Service – a managed platform with powerful capabilities for building web and mobile apps for many platforms and mobile devices. Web Apps and Mobile Apps allow developers to build web and mobile apps using languages like .NET, Java, NodeJS, PHP, or Python. In addition, Azure API Apps allows easy exposure and management of APIs, which can be accessed by mobile or web clients.
  • Power BI – a cloud-based service that provides an easy way to create rich, interactive dashboards for visualizations and analysis. Power BI also offers live dashboards, which allow users to monitor changes in the data and indicators. Power BI includes native apps for desktop and mobile devices.

Technology Choices for Optional Subsystems

Data Transformation

Data transformation allows restructuring, combination, or transformation of telemetry data.

Machine Learning

Machine learning allows for predictive algorithms to be executed over historical telemetry data, enabling scenarios like predictive maintenance.

User Management

User management supports splitting of functionality amongst different roles and users.

  • Azure Active Directory – for user management, such as authenticating user credentials, authorization of user UI capabilities, reporting and management tools users have access to, and auditing application activities.

Cross-cutting IoT Application Needs

Security

Security cuts across the entire solution, including user management and auditing, device connectivity, in-transit telemetry and at rest security.  See Azure IoT Security Architecture.

  • Azure IoT Hub Service – provides built-in secure connectivity, telemetry and event ingestion, and bi-direction communication and device management.
  • Azure Active Directory – for user management, such as authenticating user credentials, authorization of user UI capabilities, reporting and management tools users have access to, and auditing application activities.

Logging and Monitoring

Logging and monitoring used to determine health and troubleshoot failures of individual subsystems and the application as a whole.

High Availability and Disaster Recovery

High availability and disaster recovery helps you rapidly recover from systemic failures.

 

Updated on August 3, 2018

Was this article helpful?