What
is Wrong with the Way the NSF
<<=== to the home page …............ ...................... ........... ........to the publications===>>
This post has two parts, which together has lead to a paper which is currently in a preparation to by published.
Part I:
What
is Wrong with the Way the NSF
Funds Educational Projects?
The National Science Foundation has been established in 1950 with the mission of: “To promote the progress of science; to advance the national health, prosperity, and welfare; and to secure national defense; and for other purposes”: That was a quote from https://www.nsf.gov/pubs/2014/nsf14002/pdf/02_mission_vision.pdf.
Another important quote (from https://www.nsf.gov/about/what.jsp): “NSF is the only federal agency whose mission includes support for all fields of fundamental science and engineering, except for medical sciences. NSF is tasked with keeping the United States at the leading edge of discovery in a wide range of scientific areas, from astronomy to geology to zoology. So, in addition to funding research in the traditional academic areas, the agency also supports "high risk, high pay off" ideas, novel collaborations and numerous projects that may seem like science fiction today, but which the public will take for granted tomorrow. And in every case, we ensure that research is fully integrated with education so that today's revolutionary work will also be training tomorrow's top scientists and engineers”.
Sound very illumination and promising, isn’t it? But as they say, the devil is in the details.
Paraphrasing Carl Jung (and many others), any good idea when taken to the extreme becomes its own opposite.
A quote from http://www.nsf.gov/bfa/dias/policy/merit_review/facts.jsp#1: “All proposals submitted to NSF are reviewed according to the two merit review criteria: Intellectual Merit and Broader Impacts.”
The problem is that many projects in the field of education are social by their nature, do not really have strong intellectual importance, and do not intended to produce new scientific knowledge, but in order to get funded by the NSF, they have to fulfill demands of the Intellectual Merit – hence, they have to be “dressed” as a research.
A general theory of human practice recognizes three kinds of broad human practices: (a) scientific research; (b) engineering and art; (c) social advancement.
Those three practices have different goals: (a) the goal of a scientific research is discovering new knowledge; (b) for an engineering development the goal is building new devices (systems of devices), the goal of art is bringing artifacts of art; (c) the goal of a social advancement project is developing or adopting new collective practice(s) beneficial to the society (local as well as global). Since all three practices have different goals, they also should be managed differently.
Clearly, every practice has some elements of a scientific research: when we start a project – any project, we generally have some understanding of what we want to achieve and how we want to achieve that (“a hypothesis”, “methods”), and how will we assess (measure) how close we are to the goal (“results”, “facts”). But not every question is “a hypothesis”, and not every search is “a research”.
A scientific progress is the result of practices when people do something new for a large part of human culture. When that new knowledge is described, the goal of a scientific project is achieved.
A social progress is the result of innovative practices of people doing something new - for them – which they did not do in the past, even if a similar practice had been used by different people in a different place at a different time. But the goal of a social project is to make a specific societal change hear and now. When we want to induce some societal change, we have to initiate and manage a social project.
Many of the projects “imposed” on teachers are social projects by their nature, and should be treated and managed as such.
There are many things in the world which are similar on the outside but very different on the inside, or by their functions, goals, properties. For example, a space shuttle and a fighter jet look very similar, but only one can fly in the outer world (a space shuttle). The difference between a scientific research and a social project is similar to the difference between an archeological excavation and a dig for a treasure chest: they both use some digging, but the goals and the results are very different.
Below is an excerpt from an abstract of a grant proposal of a certain university (the grant received an award from the NSF): “University are conducting research on the relationship between mathematical knowledge for teaching (MKT), teaching practice, and student outcomes. … The research questions are as follows. How effective is Math Solutions as compared to a typical ad-hoc mathematics professional development? Does Math Solutions improve teachers’ MKT, the quality of their instruction, and/or their students’ outcomes? How are different aspects of teachers’ mathematical knowledge and instructions related to student achievement?”
Anyone reading the abstract will immediately understand that the project is about professional development of math teachers. The ultimate goal of the project is to train 80 fourth and fifth grade teachers.
Do the trainers need to know if the training process they plan to use will positively affect the math skills and math knowledge of the trainees?
Of course!
Does this represent a scientific problem required a scientific research?
Of course NOT!
This grant proposal represents a clear example of a social project with the goal of advancing math preparation of school teachers. But in order to get the funding from the NSF, grantees had to make it seen as a scientific project.
The research question (“How effective is Math Solutions as compared to a typical ad-hoc mathematics professional development?”) is irrelevant. All possible teacher professional development programs should be available to teachers.
As long as teachers will be able to “vote with feet” there is no need to research which program is better (assuming that the NSF will be keeping track of different programs and helping teachers to find the information on the best programs; follow to https://www.smashwords.com/books/view/665204 for more on the matter).
Instead of a “research question” applicants should had formulated “a social goal”. This is what the grantees should had written: “We want to teach math to 80 teachers; this is what the teachers will learn, and this is how we will assess the results; and for that we need 4.7 million dollars for five years (which is close to sixty thousand dollars per a preparation of a single teacher)”. The grant proposal would had held the truth, only truth, and nothing but the truth.
The NSF should not expect any big scientific outcome from a social project, but should be very demanding regarding the methods for evaluating the success of the project. In the case described above, the NSF should had demanded that applicants would guaranty that their professional development approach would definitely and visibly “improve teachers’ MKT, the quality of their instruction, and (!) their students’ outcomes” – or money back.
Instead, the NSF forces innovators (a.k.a. people producing a new social outcome, like math teachers who know math) to make them to look as scientists (a.k.a. people producing a new knowledge).
The example of a grant proposal used above to demonstrate the point of this paper is one of many grants which are social by the nature of the goals but “dressed” as scientific projects (feel free to browse at http://nsf.gov/dir/index.jsp?org=EHR; an interesting social project with the goal of improving the way the NSF funds educational projects would be analyzing education related grants issued over the last five years and classifying them into three categories – mostly scientific, mostly social, and the rest).
K-12 education is not the only one distorted by this problem. There are many projects at a college and university levels (e.g. http://www.teachology.xyz/msm.html) which are also social by their nature (with the main goal of improving some specific features of educational reality), but “dressed” as scientific ones. One of the widely spread examples is the well-known Learning Assistants program (https://learningassistantalliance.org). This program has originated as a means for fixing the shortage of STEM middle and high school teachers. One of the main premises (a.k.a. “research questions”, a.k.a. “hypotheses”) is that when undergraduate students get an opportunity to be immersed into a university teaching process, they will eventually end up teaching middle and high school students. In reality, no more than a couple percent of Learning Assistants might choose to become teachers. The program, however, might eventually have a strong social impact, because what the program really does is generating a growing number of educated and socially active people, who also have elevated awareness of what it means and how it feels to be a teacher.
In fact, almost every educational “research” project at a college or a university level is not a science project, but the one designed to keep faculty trying new things in the way faculty teaches. I say, ninety percent of issues with education college students has its roots in a high and middles school – this is where ninety percent of funding should go; when ninety percent of high school graduates will have the background sufficient to study at a college level without using any remedial courses, the most of the issues related to college education will just disappear. What really needs funding is developing standard procedures for measuring learning outcomes of college students, which (procedures) would be used by ALL colleges across the country. Until this task is finished, ANY educational project at a college or university level will not be scientific. Until standard procedures for measuring learning outcomes of college students will be developed and adopted, all college level projects will be just dressed as a “research”.
One may ask – so what? Down the road, universities get grants, teachers get taught, students get educated, everyone is happy.
The problem is that the NSF essentially forces people into faking doing science. The core of any science is being truthful about everything; including goals, methods, types of actions being used to achieve the goals. If people assume that faking science is fine – even for the sake of achieving positive social changes – that will water down the essence of science.
Such an attitude as “I do something good, so what if I pretend that I am doing science” – if not confined – might spread into other practices.
In addition to helping developing new scientific knowledge, the NSF is also working against its own mission by helping developing wrong attitude about science – at least within some projects in the field of education.
It is a scientific fact that both, the Religion and the Government, have benefited from the separation of Church and State. Similarly, the separation of programs for social advancement from programs for scientific advancement will be beneficial for both, social and scientific advancement.
Not enforcing such a separation makes the way the NSF funds of some of educational projects to be wrong.
Part II:
How much of the NSF funded “fundamental”
scientific educational research is really fundamental?
The other day I received an email from the NSF:
Here are the active links shown in the picture:
(https://www.nsf.gov/news/news_summ.jsp?cntn_id=190509&WT.mc_id=USNSF_51&WT.mc_ev=click)
The webpage tells you that (in part):
“The new awards fund projects aimed at generating foundational knowledge in:
_ Improving and advancing STEM learning and learning environments for students, parents, teachers and the general population in all settings, from formal and informal education to technological learning environments.
_ Supporting and preparing a STEM professional workforce that is ready to capitalize on unprecedented advances in technology and science and address current and future global, social and economic challenges.
_ Diversifying and increasing participation in STEM, effectively building institutional capacity and informal learning environments that foster the untapped potential of underrepresented groups in STEM fields.”
The brief reading of the bullets already raises a question – do the goals really represent the search for a fundamental scientific knowledge, or they rather aim at improving immediate social issues education system currently deals with?
The following link
leads to “the complete list of ECR projects and their abstracts” (the picture shows page 1).
The total number of projects funded within $61 million is 114. However, only 3 projects from 114 really fall in a category “fundamental scientific research”. Those three truly fundamental scientific projects are related to a neurology of thinking; they study various connections between process of thinking and processes happening in a brain while thinking. The total amount of funding set aside for those projects is #2,242,982, which is equal to 3.7 % of the total funds.
It means that 96.3 % of the funds are being used for projects of another kind (do not belong to a fundamental scientific research).
If one just reads the titles of the projects one can find several more projects which also may be sought as a part of a fundamental scientific research, but that would require the detailed analysis of the projects.
If the NSF would ask me to do such an analysis, I could, but I doubt that the NSF would.
A brief reading of the project titles and some of the abstracts shows that the majority of the projects are of a social nature; they aim at improving a current social situation by solving a specific immediate social problem within the field of education.
No doubt, some of those socially oriented projects are fundamentally important for making education better, more successful, more student oriented, more diverse.
But they would not help much to advance a science of education.
The classification of socially oriented projects as a part of a fundamental scientific research is a very common practice; and it is based on a common misconception of what a science is.
There is a wide-spread opinion (also held by many people in the field of education) that:
1) when a person poses a question, and
2) then describes some steps which would lead to the answer to this questions, and
3) then describes how he or she would assess if the question was answered correctly
– that person conducts a scientific research.
In reality, this procedure is most commonly used for achieving a specific social goal.
This procedure is used when a person feels some disconnection between his or her social position and the position the person desires to have. This procedure has been an object of a study of a General Theory of Human Activity (a.k.a. Activity Theory), which has several different forms, or academic schools, including the one used in the field of a teacher professional development.
Not any possible question (a.k.a. a proposition which starts from “Is it true that …”) should be called a hypothesis, and not any possible activity which leads to an answer should be called a research.
In general, there are three kinds of human practices/projects with the goal of advancing human life: (a) scientific research - the goal of a scientific research is discovering new knowledge; (b) engineering and art - the goal of an engineering development is building new devices (and systems of devices), the goal of art is bringing/developing artifacts of art; (c) social advancement - the goal of a social advancement project is developing or adopting new collective practice(s) (new - for the given social group, but may have been used already by other people).
Clearly, every practice has some elements of a scientific research: when we start a project, we generally have some understanding of what we want to achieve and how we want to achieve that (“a hypothesis”), and how will we assess (measure) how close we are to the goal (“facts”).
The difference between a scientific research and a social project is in “what utilizes what”.
In a scientific research, some social activity is being used as a vehicle to obtain new knowledge. In that case, some advancement in some social practice represents a “collateral” result of the research.
In a social project, some scientific knowledge is being used to achieve positive changes in a certain social situation. In this case, some newly recorded knowledge represents a “collateral” result of the project.
The majority of the 114 projects funded by the NSF aim at the achievement of some positive social changes in a certain educational environment.
For example, the very first project at the top of the first page “Transitioning Learners to Calculus in Community Colleges” aims at “Improving student outcomes in mathematics courses in community colleges”. The main vehicle of the project is improving instructions by utilizing various instruments (mostly surveys, and self-assessments). Is this an important social project? Of course! Does it represent a fundamental scientific research? Of course not!
And many more projects sound like this one. If we strip off all the scientific language, we will read – paraphrasing –
1) “We want our students to do better. For that we plan on trying this.” – if the project mostly involves faculty or teachers who directly teach students.
or
2) “We want our school teachers to teach better. For that we plan on trying this.” – if the project mostly involves faculty from a school of education.
I don’t’ claim that all projects are fall into the two described categories, but most of them do.
One might ask, what harm is in calling social projects as scientific ones? Both kinds are important and do good for education.
A short answer is: it is bad because it makes an impression of a huge amount of a scientific research happening in the field of education; when in fact a true scientific research in the field of education does not exceed 3 – 5 % of the total funding (if we want to promote a science of education to a true science we need to change that).
The bigger problem is that unwillingly “the NSF essentially forces people into faking doing science. The core of any science is being truthful about everything; including goals, methods, types of actions being used to achieve the goals. If people assume that faking science is fine – even for the sake of achieving positive social changes – that will water down the essence of science.
It is a scientific fact that both, the Religion and the Government, have benefited from the separation of Church and State. Similarly, the separation of programs for social advancement from programs for scientific advancement will be beneficial for both, social and scientific advancement.
Not enforcing such a separation makes the way the NSF funds of some of educational projects to be wrong”.
The last quote has been taken from a recent essay, which offers a broader discussion.
Another recent essay offers a discussion on what should the fundamental research in the field of education be about. The central premise of the approach for marking a research as “fundamental” is based on the facts, that
1) For every child, there is a finite number of individual characteristics describing his or her learning, behavioral, and social styles.
2) There is a finite number of subjects to learn, and within each subject there is a finite volume of knowledge to learn, and a finite number of skills to master.
Hence, it should take a finite amount of time to study all relevant and sustainable correlations (a.k.a. laws).
Tags for this web page:
Education, education reform, innovations, education innovations, physics, STEM, teachers, teacher professional development, science, science of education, data mining, educational data mining, closing a gap, school innovation, school transformation, school district, superintendent of schools, charity, philanthropy, philanthropist, non-profit for education, charitable foundation, business for education, businesses for education, business leaders, for education, Mark Zuckerberg, Zuckerberg Chan foundation, Chan Zuckerberg initiative, Bill Gates, Bill and Melinda Gates Foundation, Elon Musk, Yuri Milner, Warren Buffett, business for education, National Science Foundation, NSF.
<<=== to the home page …............ ............. ................. ...........to the publications===>>