The rapid digitization and generation of data worldwide requires new ways and technologies to capture, store, process and make it available efficiently and safely. Worldwide installed storage capacity is estimated to reach 6.8 ZB by 2020, and will grow to 8.9 ZB in 2024. Additionally, the relationship between storage at endpoints (cell phones, computers and others) will migrate to storage in data centers.

Cloud computing is a model where third parties provide flexible storage, processing and storage services; that is to say, the use of resources is scalable adjusting to the needs in a dynamic way, and with a model of payment for use of resources. Therefore, instead of buying, owning and maintaining IT infrastructure and proprietary data centers, the “pay-per-use” model is used, accessing state-of-the-art technologies, as well as the best cybersecurity standards, allowing rapid development, since the focus is on the solution and not on the restrictions imposed by proprietary technologies and infrastructure.

High Performance Computing (HPC) is the ability to process data and perform complex calculations at high speeds. To put it in perspective, a laptop or desktop computer with a 3 GHz processor can perform around 3 billion calculations per second. While that is much faster than any human can achieve, it pales in comparison to HPC solutions that can perform billions of calculations per second. One of the best known types of HPC solutions is the supercomputer, a tool that contains thousands of computer nodes that work together to complete one or more tasks. This is called parallel processing, an action very similar to having thousands of PCs networked, combining computing power to complete tasks faster.

AMAZON WEB SERVICES

The Data Observatory has as its founding partner Amazon Web Services (AWS), the leading company in the cloud computing market. AWS was launched in 2006 from the internal infrastructure that Amazon.com created to manage its online retail operations. AWS was one of the first companies to introduce a pay-as-you-go cloud computing model, which scales to provide users with computing, storage, or performance as needed. AWS currently offers various tools and solutions, which are available through the Data Observatory, among which stand out.

  1. Computing capacity. Elastic Compute Cloud (EC2) is a service that provides virtual servers, called EC2 instances, offering dozens of instance types with varying capacities and sizes, tailored to workload types and applications
  2. Storage. Simple Storage Service (S3) provides scalable object storage for data backup, collection, and analysis; Elastic Block Store provides block level storage volumes for persistent data storage when using EC2 instances; and Elastic File System offers cloud-based managed file storage;
  3. Databases, data management. The relational database service, which includes SQL options for Oracle, SQL Server, PostgreSQL, MySQL, MariaDB and Aurora, in addition to a NoSQL DynamoDB;
  4. Development tools
  5. Analytics tools