Connect with us

Science & Technology

Can State-of-the-Art Machine Learning Tools Give New Life to Household Survey Data?

Avatar photo

Published

on

In 2014 the UN called for a data revolution to put the best available tools and methods to work in service of achieving the Sustainable Development Goals. Researchers at the World Bank have responded to that call by scouring the globe for the latest machine learning tools to transform our approach to tracking progress in the fight against poverty.

“Collecting household survey data on poverty is expensive and time-consuming, which means that policy makers are often making decisions based on old data,” said Asli Demirguc-Kunt, Director of Research at the World Bank. “Machine learning could drastically change the game, making poverty measurement cheaper and much closer to real-time.”

Machine learning is a field of computer science that allows computers to examine large bodies of data to identify patterns that data scientists would never find on their own.

Olivier Dupriez, a statistician at the World Bank, unveiled results from his and colleagues’ ongoing work on machine learning and poverty measurement at a recent Policy Research Talk. According to Dupriez, machine learning could make it possible to use a handful of easy-to-collect pieces of household information—such as whether a household has basic appliances or consumes a specific type of food—to predict the poverty status of a household. This would allow for poverty estimation based on simpler household survey instruments, and, hopefully, more effective policies to end poverty.

Data scientists have been developing the foundations of machine learning over the last half century, but its use has only become commonplace with the vast increases in computing power of the last decade. Today, machine learning is employed in everything from facial recognition technology to spam filters. In the case of poverty measurement, machine learning tools are employed to examine large national household survey datasets—which can be augmented with data from other sources, such as geospatial data—to develop models that can most effectively identify poor households.

From the outset, Dupriez and his colleagues faced a challenge: academia and the private sector have already created a multitude of machine learning tools, each with its own advantages and disadvantages. One tool may do a great job of predicting national poverty rates, but perform poorly when identifying specific households that fall below the poverty line.

Given the large number of options, the first stage of research has been exploratory. Dupriez and his colleagues have used large, comprehensive datasets on household poverty from Indonesia and Malawi to test four different approaches—with widely varying results.

The first of these applied 10 out-of-the-box machine learning algorithms to predict which households in Malawi and Indonesia are poor based on a handful of available variables. None of these algorithms proved to be a clear winner; different tools performed better on different metrics of poverty prediction. However, combining the results of these 10 out-of-the-box algorithms significantly improved predictive performance on most metrics.

The World Bank researchers also crowdsourced options through a global data science competition that attracted 6,000 submissions. The five winners from Portugal, Russia, China, and the Philippines faced the challenge of predicting household poverty status using anonymized datasets; they had no idea what country the data were from or whether specific households were classified as poor. Their models slightly outperformed the best of the 10 out-of-the-box algorithms used in the first approach.

Dupriez also employed an automated machine learning process, where the computer itself identifies the optimal way to build a model of poverty prediction. This approach requires a truly high-powered computer—in this case a computer with 32 processors worked for two days straight. The model produced via automated machine learning proved to be a disappointment, however, as it fell short on most quality metrics. But this may have been the result of the task assigned to it—the computer was directed to optimize only a single metric.

Instead, the most impressive results came from pairing experts and machines. Big tech companies like Google, Netflix, and Amazon have been innovating in the area of machine learning to produce content recommendation systems. Dupriez hired outside experts (Peter Bull and Casey Fitpatrick of DrivenData), who deployed an algorithm similar to the one Google uses for its Play Store. This algorithm produced excellent results on almost all metrics, making it a strong contender for additional investigation despite its complexity.

While the results so far are promising, the research is still in an early phase, with much more work needed to turn machine learning into a practical tool that can make poverty measurement faster and cheaper. Globally, data scientists need to commit to openness around data, software, and methods to make this a reality.

“The reason that our outside experts were able to build the best performing model is that they know a lot about what model works with what kind of data and what kind of challenge,” said Dupriez. “We need to develop a library of reproducible scripts and open data where researchers can find any kind of machine learning model, no matter who created it, so that we as a community can apply it to the most pressing challenges the world faces.”

World Bank

Continue Reading
Comments

Science & Technology

The Promise of Blockchain in Mega Sport Events

Published

on

Authors: Dr. Aiman Erbad and Dr. Mohamed Abdallah

Amid the excitement and anticipation of the FIFA World Cup Qatar 2022TM, sport remains a business. Like other global industries, the adoption of technology innovations is driving greater efficiency and transparency to generate benefits for sports organizations, leagues, clubs, and fans.

Researchers at the College of Science and Engineering (CSE), Hamad Bin Khalifa University (HBKU), make the case for adopting blockchain-powered solutions in delivering seamless sport mega events by outlining some of the top use cases.

Understanding blockchain

“Blockchain can solve many real-world problems,” explains Dr. Mohamed Abdallah, Associate Professor in the Division of Information and Computing Technology (ICT) at CSE.

“For mega sport events, the benefits can be exceptional. Because of its transparent data structure, blockchain can facilitate secure and reliable data exchange at the individual, institutional, or national systems level as needed, without the need for intermediaries to ensure mutual trust and the authenticity of the data exchanged.”

The chaos of the UEFA Champions League final between English team Liverpool and Spanish club Real Madrid in May 2022, which resulted from the illegal distribution of non-validated tickets, is likely to have accelerated the recognition of blockchain’s benefits for the sport industry. The ensuing government inquiry unequivocally called for using blockchain for ticketing to prevent a similar fiasco at future events. A closer look at the nature of this cutting-edge technology reveals why.

How blockchain works

By its functional nature, a blockchain is a distributed (or shared) digital ledger that stores encrypted blocks of transaction data securely chained together in chronological order. Unlike other ledgers or databases, blockchain combines unique security features based on cryptographic techniques and its chronological chain structure.

In its standard form, blockchain provides immutability (data entered is permanently recorded), transparency (data is visible to everyone involved), and decentralization (all computers in the network have a copy of the blockchain to collectively maintain control). These features facilitate a tamper-proof, reliable way of storing, exchanging, and tracking information.

A key use case for mega events: preventing ticketing scandals

Dr. Abdallah and Dr. Aiman Erbad, Associate Professor and Head of ICT at CSE, add their expert voices to arguments that the UEFA Champions League final chaos could have been prevented using a blockchain platform with a self-enforcing contract capability to facilitate a secure ticket purchase process.

In practical terms, tickets can be stored on the blockchain denoted with unique cryptographic tokens. Each ticket can be linked to the authentic owner, providing traceability and accountability that prevents forgery. In this way, it can effectively reduce the impact of bots and/or scammers buying large numbers of tickets for illegal resale.

Using blockchain-based “smart contract” technology, ticketing entities can set the required resale rules to ensure a fair and secure market. These digital contracts can facilitate transactions between buyers and sellers while maintaining data accountability and traceability.

A related use case is storing the chain of ticket ownership. These records cannot be forged since changes are verified and tracked, ensuring data integrity. It can help customers validate the authenticity of tickets to avoid being trapped by ticketing scams.

Other use cases in sport

Blockchain-powered fan engagement is a growing use case for the sports industry. Several professional leagues and clubs are using blockchain to establish trustworthy fan databases that facilitate the distribution of “fan tokens”. With the status of a digital asset (created on a blockchain), the tokens can be redeemed by fans for rewards such as VIP experiences or ticket promotions. The increased fan engagement can potentially create new revenue streams for clubs; for example, incentivizing them to attend more events in person. Fan tokens have been rolled out by professional sports teams all over the world, including Paris Saint-Germain and FC Barcelona.

In another use case, the market for sports collectibles and memorabilia can leverage blockchain to establish trust and traceability. Experts have warned that fraud is rampant in the sports collectibles and memorabilia market. Blockchain can ensure the authenticity of special items through the use of digital identities.

CSE’s own blockchain-based applications

CSE faculty members are developing innovative use cases for blockchain in a range of applied settings.

“Our research focuses on the applicability of blockchain in solving real-world problems, such as securing data access in healthcare and decentralized trading,” says Dr. Erbad.

“We also study the technical aspects of blockchain to enhance its security, privacy, and efficiency. We have investigated the possibility of reducing energy consumption in public blockchains and developed an energy-efficient consensus algorithm. In other areas, we have also investigated using artificial intelligence in combination with blockchain smart contracts, called Rational Contracts, to provide smart resource trading with optimal prices in smart city applications.”

Among CSE’s blockchain-based applications are a trading platform for electric vehicle charging in smart cities, a decentralized ride-sharing service, a privacy-preserving decentralized stock exchange platform, a scalable energy trading sealed-bid auction mechanism, real-time secure health data exchange system, and a cooperative spectrum management system for 5G networks.

A national blueprint for Qatar

CSE had a leading role in developing the Qatar National Blockchain Blueprint in collaboration with the Communications Regulatory Authority and Qatar University. The blueprint highlights how blockchain can advance Qatar’s innovative and growing IT sector.

Essential blockchain requirements and recommendations for building a solid regulatory framework drive its pivotal goal of facilitating blockchain’s adoption at the national level, in support of Qatar National Vision 2030 and Qatar National Development Strategy. To achieve this, the blueprint outlines the conditions and incentives each sector must provide for the level of technology adoption needed to allow start-ups, pilot projects, and new companies to emerge. The strategy is an important step for Qatar, its sports, and other leading industries, to reap the societal benefits of this innovative technology.

For more information on the work of the College of Science and Engineering, please visit cse.hbku.edu.qa. To know more about Qatar National Blockchain Blueprint, please visit: https://www.cra.gov.qa/document/national-blockchain-blueprint

Continue Reading

Science & Technology

Interesting archaeological discovery in Israel

Avatar photo

Published

on

An ancient scarab from three thousand years ago was surprisingly discovered during a school trip to Azor, near Tel Aviv, Israel. The scene depicted on the scarab probably represents the conferral of legitimate power and authority on a local ruler.

“We were wandering around, when I saw something that looked like a small toy on the ground,” told Gilad Stern of the Education Centre of the Israeli Antiquities Authorityntre, who was leading the school trip. “An inner voice told me: ‘Pick it up and turn it over.’ I was amazed: it was a scarab with a clearly engraved scene, the dream of every amateur archaeologist. The pupils were really enthusiastic!”.

The visit of the Rabin Middle School eight graders took place as part of a tour guide course organised by the Education Centre of the Israel Antiquities Authority for the third consecutive year. The course enables students to teach the residents of Azor about the local archaeological heritage.

The scarab was designed in the shape of the common dung beetle. The ancient Egyptians saw in the gesture of the tiny scarab, which rolls a ball of dung twice its size where it stores its future offspring, the embodiment of creation and regeneration, similar to the gesture of the Creator God.

According to Dr. Amir Golani, an expert of the Israeli Antiquities Authority specialized in the Bronze Age period, “the scarab was used as a seal and was a symbol of power and status. It could be inserted into a necklace or a ring. It is made of silicate earthenware covered with a bluish-green glaze. It could have fallen from the hands of an important and influential personage passing through the area, or it could have been deliberately buried in the ground with other objects and after thousands of years returned to the surface. It is difficult to determine the precise original context.”

The lower, flat part of the scarab seal depicts a figure seated on a chair in front of a standing figure, whose arm is raised above that of the seated person. The standing figure has an elongated head, which seems to represent the crown of an Egyptian pharaoh. It is possible that we are seeing here a snapshot of a scene in which the Egyptian pharaoh confers power and authority on a local Canaanite.

“This scene fundamentally reflects the geopolitical reality that prevailed in the Land of Canaan during the Late Bronze Age (approx. 1500-1000 BC), when local Canaanite rulers lived under Egypt’s political and cultural hegemony (and sometimes rebelled against it)” – said Dr. Golani. “It is therefore very likely that the seal dates back to the Late Bronze Age, when the local Canaanites were ruled by the Egyptian Empire”.

Scarab seals are indeed distinctly Egyptian, but their widespread use extended beyond the borders of ancient Egypt. Hundreds of scarabs were discovered in the Land of ancient Israel, mostly in tombs, but also in settlement layers. Some of them were imported from Egypt, many others were imitated in ancient Israel by local craftsmen under Egyptian influence. The level of workmanship of the particular scarab found is not typical of Egypt and may be a product of local craftsmen.

Continue Reading

Science & Technology

Towards Efficient Matrix Multiplication

Avatar photo

Published

on

Algorithms have, over the years, helped mathematicians/scientists solve numerous fundamental operations. From the early use of simple algorithms by Egyptian, Greek, and Persian mathematicians to the shift towards more robust AI-enabled algorithms, their evolution has manifested incredible progress in the technological realm. While Artificial Intelligence (AI) and Machine Learning (ML) are extending their reach and contributions in various military and civilian domains, it is interesting to witness the application of the technology on itself, i.e., using ML to improve the effectiveness of its underlying algorithms.

Despite the increased familiarisation with algorithms over time, it remains fairly strenuous to find new algorithms that can prove reliable and accurate. Interestingly, ‘Discovering faster matrix multiplication algorithms with reinforcement learning,’ a recent study by DeepMind, a British AI subsidiary in London, published in Nature, has demonstrated some interesting findings in this regard. It revealed new shortcuts simulated by AI for faster mathematical calculations vis-à-vis matrix multiplication.

DeepMind developed an AI system called ‘AlphaTensor’, to expedite matrix multiplication. Matrix multiplication – which uses two grids of numbers multiplied together – is a simple algebraic expression often taught in high school. However, its ubiquitous use in the digital world and computing has considerable influence.

‘AlphaTensor’ was tasked with creating novel, correct, and efficient algorithms to carry out matrix multiplication with the least number of steps possible. The algorithm discovery process was treated as a single-player game. It used AlphaZero – the same AI agent which gained global traction when it displayed extraordinary intelligence in board games like Chess and Go.

AlphaTensor conceptualised the board into a 3-D array of numbers which, through a limited number of moves, tried to find the correct multiplication algorithms. It uses reinforcement learning, where the neural networks interact with the environment toward a specific goal. If the results are favourable, the internal parameters are updated. It also uses Tree Search, in which the ML explores the results of branching possibilities to choose the next action. It seeks to identify the most promising action at each step. The outcomes are used to sharpen neural networks, further helping the tree search, and providing more successes to learn from.

As per the paper’s findings, AlphaTensor discovered thousands of algorithms for various sizes for multiplication matrices, some of which were able to break decades-long computational efficiency records of the previously existing algorithms. They overshadowed the towering complexity of the best-known Strassen’s two-level algorithm for multiplying matrix. For example, AlphaTensor found an algorithm for solving a 4 x 4 matrice in 47 steps overperforming the Strassen algorithm, which used 49 steps for the same operation. Similarly, if a set of matrices was solved using 80 multiplication steps, AlphaTensor reduced it to only 76 steps. This development has caused quite a stir in the tech world as it is being claimed that a fifty-year old record has been broken in Computer Science.

However, the episode underlines some important implications. Given that matrix multiplication is a core component of the digital world, companies around the world have invested considerable time and resources in computer hardware for matrix multiplication. Since it is used across a wide range of domains, including computing, processing images, generating graphics, running simulations, digital communication, and neural networks etc. – to name a few, even minor improvements in matrix multiplication’s efficiency could have a notable and widespread impact in the concerned fields.

The findings manifest the potential of ML to solve even more complicated mathematical problems. The automatic discovery of algorithms via ML offers new capacities to surpass the existing best human-designed algorithms. It introduces new ML techniques, which have the potential to increase computing speed by 20 percent leading to much more feasible timelines. It is pertinent to mention that a lesser number of operations lead to not only lesser time but also less amount of energy spent.

The finding has presented a model to gamify ML to solve mathematical operations. It exhibited that AlphaZero is a potent algorithm that could be used beyond winning traditional games and be applied to solving complex mathematical operations/tasks.

This DeepMind discovery can pave the way for future research on understanding matrix multiplication algorithms and be an inspiration to use AI for algorithm discovery for other computing tasks and set the stage for a possible breakthrough in the field. 

The increased efficiency of matrix multiplication has once again brought into light the ever-expanding potential of AI. To be fair, such developments do not infer that human programmers would be out of the job soon; rather, at least for now, it should be seen as an addition of an optimisation tool in the coder’s arsenal, which could lead to more innovative discoveries in the future with remarkable implications for the world.

Continue Reading

Publications

Latest

Trending