Connect with us

Tech

Can State-of-the-Art Machine Learning Tools Give New Life to Household Survey Data?

MD Staff

Published

on

In 2014 the UN called for a data revolution to put the best available tools and methods to work in service of achieving the Sustainable Development Goals. Researchers at the World Bank have responded to that call by scouring the globe for the latest machine learning tools to transform our approach to tracking progress in the fight against poverty.

“Collecting household survey data on poverty is expensive and time-consuming, which means that policy makers are often making decisions based on old data,” said Asli Demirguc-Kunt, Director of Research at the World Bank. “Machine learning could drastically change the game, making poverty measurement cheaper and much closer to real-time.”

Machine learning is a field of computer science that allows computers to examine large bodies of data to identify patterns that data scientists would never find on their own.

Olivier Dupriez, a statistician at the World Bank, unveiled results from his and colleagues’ ongoing work on machine learning and poverty measurement at a recent Policy Research Talk. According to Dupriez, machine learning could make it possible to use a handful of easy-to-collect pieces of household information—such as whether a household has basic appliances or consumes a specific type of food—to predict the poverty status of a household. This would allow for poverty estimation based on simpler household survey instruments, and, hopefully, more effective policies to end poverty.

Data scientists have been developing the foundations of machine learning over the last half century, but its use has only become commonplace with the vast increases in computing power of the last decade. Today, machine learning is employed in everything from facial recognition technology to spam filters. In the case of poverty measurement, machine learning tools are employed to examine large national household survey datasets—which can be augmented with data from other sources, such as geospatial data—to develop models that can most effectively identify poor households.

From the outset, Dupriez and his colleagues faced a challenge: academia and the private sector have already created a multitude of machine learning tools, each with its own advantages and disadvantages. One tool may do a great job of predicting national poverty rates, but perform poorly when identifying specific households that fall below the poverty line.

Given the large number of options, the first stage of research has been exploratory. Dupriez and his colleagues have used large, comprehensive datasets on household poverty from Indonesia and Malawi to test four different approaches—with widely varying results.

The first of these applied 10 out-of-the-box machine learning algorithms to predict which households in Malawi and Indonesia are poor based on a handful of available variables. None of these algorithms proved to be a clear winner; different tools performed better on different metrics of poverty prediction. However, combining the results of these 10 out-of-the-box algorithms significantly improved predictive performance on most metrics.

The World Bank researchers also crowdsourced options through a global data science competition that attracted 6,000 submissions. The five winners from Portugal, Russia, China, and the Philippines faced the challenge of predicting household poverty status using anonymized datasets; they had no idea what country the data were from or whether specific households were classified as poor. Their models slightly outperformed the best of the 10 out-of-the-box algorithms used in the first approach.

Dupriez also employed an automated machine learning process, where the computer itself identifies the optimal way to build a model of poverty prediction. This approach requires a truly high-powered computer—in this case a computer with 32 processors worked for two days straight. The model produced via automated machine learning proved to be a disappointment, however, as it fell short on most quality metrics. But this may have been the result of the task assigned to it—the computer was directed to optimize only a single metric.

Instead, the most impressive results came from pairing experts and machines. Big tech companies like Google, Netflix, and Amazon have been innovating in the area of machine learning to produce content recommendation systems. Dupriez hired outside experts (Peter Bull and Casey Fitpatrick of DrivenData), who deployed an algorithm similar to the one Google uses for its Play Store. This algorithm produced excellent results on almost all metrics, making it a strong contender for additional investigation despite its complexity.

While the results so far are promising, the research is still in an early phase, with much more work needed to turn machine learning into a practical tool that can make poverty measurement faster and cheaper. Globally, data scientists need to commit to openness around data, software, and methods to make this a reality.

“The reason that our outside experts were able to build the best performing model is that they know a lot about what model works with what kind of data and what kind of challenge,” said Dupriez. “We need to develop a library of reproducible scripts and open data where researchers can find any kind of machine learning model, no matter who created it, so that we as a community can apply it to the most pressing challenges the world faces.”

World Bank

Continue Reading
Comments

Tech

World’s Best Blockchain and Artificial Intelligence based Idea for the Financial Markets

Osama Rizvi

Published

on

Some saw Blockchain only in the form of a cryptocurrency and moreover Bitcoin. The purpose behind their understanding was “easy money”. Few won, many lost. Then there were others who could see, in Blockchain, a completely different, revolutionizing technology that would not help in tackling the problem of bio-piracy and change the shipping industry for good. Evidently, the latter were the winners. All of them. As they were able to grasp the true essence of what will be the future.

Welcome to the world of investment and finance. Riddled with speculation, inside information, uncertainty. Hordes of amateur investors due to the lack of information or rather inaccessibility to the right information are drawn to different investment opportunities in different asset classes, only to lose money later on. But that might change very soon!

What is the biggest problem all investor face? Unpredictability. What creates this unpredictability? Reasons galore.One of them is that large financial institutions prefer to serve the needs of the institutional clients over small investors. Second, over a billion small investors spread around the world, in any financial market, ranging from stocks to bonds to commodities to forex and now the newly created asset class of crypto currencies and Tokens, have to rely on the advice of a few human beings, called analysts, whose advice can go wrong at times. And events like financial crisis of the past only cement this fact.

Here we should divert our attention to Artificial Intelligence which in the true and purest sense of the word is revolutionizing the way we are living and will live in the future. From weapon’s industry to health, and from lawyers to analysts everyone is using AI to deliver their products or services in a more efficient and effective way. So, why not combine Blockchain and AI together to create a platform that provides with a better direction in the field of financial markets? A team of 4 companies claim that they have successfully concocted the above two technologies and come up with a platform: NAi Tech.

When the platform goes live over a billion people can benefit. This formidable combination can assist ,novice or veteran investors alike,in predicting the price movement of their investment class, in any direction, along with the phenomenon called “Wisdom Of The Crowd”, which dictates that the probability of any even happening in the financial markets is somewhat dependent on what the collective opinion of the investors is. Blockchain, Artificial Intelligence and “Wisdom of the Crowd” on one platform will change the future of investment decisions for an untapped market of over one billion investors around the world.

NAi Technologies Europe,which is a London registered corporation, is determined to provide the world with such a platform.With a partnership of 4 companies around the world, 250 plus staff in 4 countries and a team consisting of Blockchain PhD’s and Artificial intelligence experts and financial analysts, NAi tech is destined to change the world of investing for many!

Meet the Team: Young energetic founders of this partnership with a proven track record in the technology industry and over 900 delivered projects in the past 10 years are excited to launch this platform and serve the needs of the investors. The President of this partnership, Malik Mudassir, a veteran of the tech industry and founder of Appsgenii, a 10 year old development firm with over 400 projects delivered internationally along with Shahmir Amir, Founder of Dubai based Veiliux, the world number one cyber security prodigy, a 23 year old young man who has been sought by Facebook, Yahoo, Air BNB, Google, Amazon and 120 top tech firms around the world are confident that they will be able to deliver their dream project along with their 250 plus team. Under the tutelage of people like Dr. Atif Farid, a Senior Data scientist with specialization in Blockchain, AI and machine learning with experience in institutes like Bank of America and Dr. Ahmed Hamdhan with over 20 years of experience of financial services and a Senior Program Director at Microsoft HQ in US, the team has only got stronger.

“This is a project that is the need of a billion plus investors around the world. With Artificial Intelligence, you are landing satellites on other planets and gathering big data to predict human behaviors, the most difficult to predict, why then can’t we do the same for the financial markets. ?” says Shahmir Amir, from his office in Dubai on a telephonic interview.

It is instructive to quote an example of Augur here, which is Blockchain-based prediction software. In its ICO it raised 19,053.92 BTC! The fact that Augur doesn’t encompasses that extent of the market (it only focuses on Sports)that NAi plans to do is promising as to the potential of the later.

When online this platform is expected to attract a lot of attention. Given its promise and the way it is going to work. It is not hard to see why!

Continue Reading

Tech

For the greater good? The loss of jobs in the digital era

Published

on

The year is 2040. Drones buzz over neighbourhoods, delivering packages. Smart homes, with interconnected Wi-Fi devices, eliminate the need for housework. Driverless vehicles take us from A to B at great speed. Wars are still fought but digitally, with lines of code and armies of robots. We vacation in space, and share stories about the moon.

In this intelligent machine age, what role will we play? Some reports, examining the implications of the digital revolution for labour markets, are forecasting a bleak future.

The concerns relate to the potential for labour displacement, as systems of artificial intelligence and automation gain increasing traction in the workplace. As these systems evolve and become ever more sophisticated, the argument goes that they will be able to outperform humans, offering greater degrees of precision, efficiency, competitiveness and reliability. Over time, a larger share of our operations is likely to be outsourced to machines.

Does this hypothesis have merit? Will capital soon no longer be able to cohabit in harmony with labour? Should we be concerned about the prospect of mass ‘technological unemployment’?

The man vs. machine debate is centuries-old. John Maynard Keynes first popularised the term ‘technological unemployment’ in his 1930 essay Economic Possibilities for our Grandchildren. Keynes regarded the phenomenon as a “temporary phase of maladjustment” for countries at the frontier of progress. On the other side of the debate, techno-pessimists, such as the classical economist, David Ricardo, instead, believed that the introduction of new technologies could lead to a sustained decline of the working population.

To understand which argument aligns better with today’s technological and labour market landscape, let’s consider some recent developments.

It is undeniable that the world and our role within it is rapidly changing. Just look at the staggering developments taking place in the transportation sector. In the Jetsons, an animated sitcom which first aired six decades ago, the inhabitants of an imaginary future commuted to work in flying cars. Today, we are on the brink of turning that vision into reality. UBER has plans to establish an aerial taxi service by 2023, and other companies have already developed flying car prototypes. Many projects under development today weren’t even anticipated by the science fiction of the past. For instance, Elon Musk, the man behind both Tesla and SpaceX, is building an underground network of tunnels that run many layers deep across the eastern United States, to transport cars and alleviate congestion challenges. In addition, in several countries, driverless cars are currently being tested. Automakers anticipate that fully-autonomous vehicles will be chauffeuring us around within the next three years.

These are just a small selection of the numerous examples of comprehensive transformation taking place today. But will we really benefit from such change? We have to wonder whether there is some irrational exuberance.

The long view of innovation, however, provides good reason for optimism. During each era of revolutionary change, innovation has lifted productivity, reduced the prices of goods and services, created new industries, stimulated output and generated fresh employment opportunities.

The first industrial revolution brought with it the power of steam and machine-based manufacturing. The new industries and jobs it generated more than offset the displacement of skilled workers producing hand-made goods. The advent of the automobile in the 19th century did the same, relative to the jobs that were lost from the horse and carriage economy. More recently, the silicon revolution gave us the power of computing, and the internet. These technologies created new businesses, tore down geographical barriers and massively disrupted the ways in which we interact. Like those that preceded it, the silicon revolution, generated far more jobs than were lost, for example in basic administrative operations.

In other words, the available body of empirical evidence indicates that short-term labour displacement, arising from technological change, has always been more than offset by the expansion of labour markets in the long-term. There is also some evidence of a similar pattern taking shape today. Since the global financial crisis the rate of unemployment has fallen sharply, and the main reason behind this decline has been very strong rates of new job creation. In the UK, technology has recently contributed to the loss of 800,000 jobs but has helped to create at least 3.5 million jobs. Each of these jobs is paying, on average, almost £10,000 more per annum compared to those that have been lost. Business sentiment, additionally, remains largely positive regarding the impact of technology on labour markets. A recent survey, undertaken by KPMG, of chief executive officers (CEOs) in the UK, reveals that seventy-one per cent believe that artificial intelligence will create more jobs than it destroys.

OK, let’s pause for a bit.

The past is not always a reliable indicator of the future. So could this time be different? There is reason to think so. Technological change is progressing at an unprecedented rate. New advancements are taking place almost daily, and their diffusion into the workplace is accelerating.

Last year, over 40 per cent of adults in the UK managed their bank accounts using smartphones. Within the next five years, this figure is projected to rise to 70 per cent, reflecting increasing numbers of mobile users in rural areas. By that time, analysts believe that customers will only visit their bank only twice a year. These trends have driven a heavy consolidation of banks around the world. In 2017, major UK banks shut, or announced plans to shut, nearly 1,000 branches. Thousands of jobs have already been lost.

A shift to driverless vehicles, likewise, could impact significant numbers of people, from lorry drivers to bus drivers to the various constituents of the gig economy. In the UK alone, over a half million people are currently employed in road transportation. Relative to earlier anxieties regarding the potential of systems like UBER to reduce jobs for ‘black cab’ drivers, these new developments surely provide greater grounds for unease.

Workers in the fast food industry could also be at risk, owing to technologies that enable self-service. McDonald’s, for instance, recently piloted “create your taste” touchscreens in its US-based restaurants. Through this system, customers could craft their own burger, and place orders at the touch of a button. The need for human interaction was eliminated. In America alone, almost 4 million people are currently employed in fast food restaurants.

Even recruiters are finding themselves threatened. Based on social media activity, work tenure, and purchasing history, algorithms can now predict when someone will be ready for a job. Text analysis can identify skills and experience many times faster than humans can. As a result, some estimates are giving the existing HR recruitment industry two to four years more at best. Hiring, for now, will still require a human touch. But that may change over time too. It is not implausible to imagine software capable of assessing personality, which scrutinises candidates on factors such as tone, facial movements and body language.

The list of impacted industries goes on and on and on. All are in the same boat.

So was Keynes right, or was Ricardo? Before we jump to conclusions regarding the nature of the relationship between technological innovation and labour markets, let’s try a little thought experiment. Take it as given that, in line with empirical evidence, the disruption being observed in labour markets today will in the future be overshadowed by an expansion in output and jobs. That being the case, would you be prepared to forego your employment now to enable a higher standard of living for your children and your grandchildren tomorrow?

If the evidence checks out, then our view on technology and the value of innovation really boils down to this one question.

Source: Commonwealth

Continue Reading

Tech

How Artificial Intelligence Uses Social Media Data to Machine Humanity

Published

on

In the process of artificial intelligence development, AI+ all big data seems to be able to increase efficiency and get a good result. As a matter of course, the combination of social media data and AI will have a negative impact.

We are used to expressing emotions, expressing interests and hobbies in social media, and even inadvertently revealing our personal information. As an ordinary person, even if you know that this is an open space, you will not have the impression of data leakage. After all, most of the information display weights of social media are based on the popularity of information, and individual users’ ordinary speeches are difficult to be unfamiliar. People deliberately see.

But with the super power and data analysis capabilities that AI brings, everything becomes different. After crawling data, AI+ social media data can see more than just one person’s reaction, emotion and expression to things, but can see information about a whole group or even a whole ethnic group. The previous series of information disclosures that almost ran through Facebook’s doorway revealed how much of the scattered personal information that AI might have had.

However, social media data is not only dark, but there are already many neurological and psychological studies that have begun to use AI and social media data to use machines to deal with human nature.

“Twitter big data tells you that people around the world are the same”

Recently, the University of Bristol used machine learning to analyze 800 million tweets in 57 cities in the UK within four years and reached a conclusion that we have long pondered about. Humans generally have high emotions in the morning and low emotions in the night.

The whole analysis process is like this. The research team sampled through the Twitter search API and collected 800 million tweets. All the # topics, emojis, holiday greetings, etc. are washed away, and the words are tagged according to the psychometric method.

For example: emotional positive emotions and negative emotions; time-oriented attention now, attention to the past, attention to the future; personal attention to work, family, money, society, religion, and so on.

With this strictly machine-learning model based on the dimensions of psychological research, the research on social media data will be more specialized, rather than simply relying on NLP to analyze the emotions in the language.

The conclusion of the final study is: In the 24 hours of the day, human emotions will not only change, thinking patterns will also change.

From the 5-6 pm of the day, people began to enter the peak period of expression in social media, and at this time people’s emotional expression is more positive, and the focus is also more on the individual status. As time goes by from 7-9, people’s emotions tend to be biased towards anger, but if it is on non-working days, this positive and enjoyable state will continue.

At this time, people’s thinking patterns tend to be class-thinking, thinking is more clear and direct, logical, and stereotyped tendencies appear.

At night, people’s emotional expression will turn negative, and the focus will shift from individuals to society. As time goes on, the closer to the next day’s 3-4 a.m., the more people focus their attention on religion. In this period of time, people’s thinking patterns tend to be existentialism, which reflects the state of confusion, anxiety, irrationality, and willingness to participate and share.

In plain words, one’s general state is to rise in the morning with enthusiasm and self-confidence, to plan his own life in the blood of chickens, and to wait until the evening when he starts to fall into a state of whimper, paying attention to every corner of the world. People who are sad or touched, if they can’t sleep at night, begin to seek religious salvation. Look, is this process the same for Chinese foreigners?

When social media becomes a research assistant in psychology, can it be diagnosed from a selfie?

In fact, the time-cycle changes in human emotions have long been confirmed. Because of the physiological causes of nerve fatigue, melatonin secretion, etc., our emotions will show different states during the day.

Although this study of social media big data just confirmed this change again and did not uncover more reasons for emotional change, it is the first time that the relationship between mood cycles and thinking patterns has changed. In fact, there are many researches on social media data and psychology, and many interesting information have been discovered.

For example, last year, the University of Pittsburgh conducted a survey to investigate the social media use of depression patients. The results showed that the average duration of social media usage for depression patients was much higher than that of ordinary people.

Harvard University research shows that people with depression prefer to use cool, faded, or black-and-white filters when they publish photos on social media.

Unbreakable Ethics Levels

At present, the role of social media data for psychology seems to remain in academic research. Can we see the psychological application of social media data in our lifetime?

At present, social media data has at least the following applications for psychology:

1.As an aid to the measurement of mental state

In addition to those mental illnesses that directly lead to hallucinations, insomnia, and other direct manifestations, there are many types or degrees of mental illness that are difficult to objectively feel. Most of the time, it is necessary to rely on the face-to-face consultation or to fill in the psychological state measurement table to confirm, but the patient may not be able to directly show his or her true state when he or she fills out independently. At this time, information undoubtedly revealed in social media can be used as a support.

2.The psychological status of the group

Compared with the individual’s psychological problems, the more complicated situation is experienced by an entire group. For example: changes in mental state that can occur when disasters or accidents occur.

For example, employee/student suicide occurs in a company or school, or an entire region suffers serious natural disasters such as earthquakes and typhoons. At this time, we often do not have the energy to do psychological counseling for everyone, and there is no way to assess the psychological status of the group as a whole. At most, the group conducts psychological counseling in the form of group lessons.

At this time, using machine learning to research social media data, you can clearly see the group’s psychological response to events. Even the long-term psychological status tracking of the crowd, and selective, targeted psychological counseling.

HIT has proposed a method to identify college students’ social media data by establishing classifiers to identify the risk of depression.

In fact, the application methods mentioned above are hardly technically difficult to achieve. Although the results obtained may not always be absolutely accurate, the value that can be provided for psychology, a labor-intensive industry, is very small.

But the biggest issue is whether it is ethical. Should publicly released social media data be considered personal privacy? The information extracted from it is not considered personal privacy? Even if it is a patient with mental illness, citizens should have the right not to disclose their prevalence, and to discover the citizens’ mental health status through social media data. Is this a serious violation of this power? In particular, if this technology is applied to colleges and universities, will anyone be so concerned that the teachers and classmates around them have learned their psychological state and have made their mental condition worse?

In fact, to a certain extent, we sometimes deliberately choose some ineffective solutions to problems, but we can exchange security and freedom for the soul.

Continue Reading

Latest

Trending

Copyright © 2018 Modern Diplomacy